feat: explicit environment variables
#15934
Closing issue
Describe the problem
I often need both a private and a public env var on the server, so I have to do something like this which is a bit tedious:
import { env as privateEnv } from "$env/dynamic/private";
import { env as publicEnv } from "$env/dynamic/public";
Describe the proposed solution
Either include the public env vars in $env/dynamic/private, or create common import { env } from "$env/dynamic" that is only usable on the server and includes both private and public env vars.
Alternatives considered
No response
Importance
nice to have
Additional Information
Related: #8474
Pull request
Today, environment variables are implicitly available: anything in process.env can be imported directly (as long as the name is a valid identifier) from $env/static/private, or accessed via env imported from $env/dynamic/private. Environment variables beginning with publicPrefix are available from the public counterparts.
This is... fine, but there is some room for improvement:
- You get random auto-import suggestions — if I type
Hto begin importingHeader.svelte, my editor will helpfully suggest that I meantHOMEBREW_CELLARor something else improbable - There's inconsistency between static (named imports) and dynamic (
env.BLAH), which feels weird. If for some reason you use both$env/dynamic/publicand$env/dynamic/privatein the same module, you will need to rename them since you can't have two imports calledenv - It's weird that you can reference a given env var both statically (inlined into the build) and dynamically (evaluated at runtime)
- There's a lot of modules!
- There's no central place that describes which environment variables are expected
- No validation — if a critical env var is missing or malformed, you might not find out until you start seeing errors in your production logs
- No type safety or inline documentation. Everything is just a string whose purpose you have to infer from the name
This PR presents an explicit alternative. Opt in by adding the following flag to your svelte.config.js...
export default {
kit: {
experimental: {
explicitEnvironmentVariables: true
}
}
};
...and creating a src/env.ts (or src/env.js) that exports variables:
import { defineEnvVars } from '@sveltejs/kit/hooks';
import * as v from 'valibot';
export const variables = defineEnvVars({
// by default, env vars are a) secret, b) evaluated when the app starts, and c) required
POSTGRES_URL: {},
// some environment variables are safe to be used in the client
GOOGLE_ANALYTICS_ID: {
public: true,
// env vars can be validated, using any https://standardschema.dev library
// (here, we're using https://valibot.dev)
schema: v.pipe(v.string(), v.regex(/G-[A-Z0-9]+/))
},
// some can be evaluated at build time, meaning they can be used for dead code elimination
SHOW_DEBUGGING_OVERLAY: {
public: true,
static: true,
// if a description is provided, it will be visible when the imported variable is hovered
description: 'If enabled, will show a FPS meter in the corner of the page',
// validators can transform inputs, e.g. to a boolean
schema: v.pipe(
v.optional(v.string(), ''),
v.transform((str) => str !== '')
)
}
});
The
defineEnvVarshelper just returns its argument, but is useful as it enforces the correct types (and provides autocomplete, etc). I'm tempted to add similar helpers for otherhooks, though that's a conversation for another time.
With those variables defined, we can import them via two new modules — $app/env/private and $app/env/public. As with existing server-only modules, $app/env/private cannot be imported into code that can run in the client.
Types are inferred from validators, where provided, and inline documentation is attached to the variables:
If environment variables are missing, the app errors on startup (or build, for static env vars).
The validators (and indeed the entire src/env module) only ever run on the server. Public variables are sent to the client with the initial server rendered HTML, or (if the initial document is prerendered) by importing a generated module, the same as happens today.
The $app/environment module is aliased as $app/env, to make everything feel a bit more cohesive. The goal is to remove $app/environment and the four $env/* modules (and the associated configuration) in SvelteKit 3.
TODO:
-
generateEnvModuleforadapter-static - Populate
%sveltekit.env%with this rather than the existing env vars, where appropriate - I just realised that public static vars don't work as advertised right now — need to fix that
- Don't bother sending static vars from the server to the client
- Make the docs nice
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:
- It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
- This message body should clearly illustrate what problems it solves.
- Ideally, include a test that fails without this PR but passes with it.
Tests
- Run the tests with
pnpm testand lint the project withpnpm lintandpnpm check
Changesets
- If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running
pnpm changesetand following the prompts. Changesets that add features should beminorand those that fix bugs should bepatch. Please prefix changeset messages withfeat:,fix:, orchore:.
Edits
- Please ensure that 'Allow edits from maintainers' is checked. PRs without this option may be closed.
Info
🦋 Changeset detected
Latest commit: ab0db74
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 1 package
| Name | Type |
|---|---|
| @sveltejs/kit | Minor |
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
There's one small detail remaining, which is that $app/env/public can't currently be loaded in a service worker. I think it'd be okay to handle that in a follow-up PR.
Super awesome.
I think I would love to have one more thing here - a decode option. It would be a function that returns the decoded value, or perhaps a value of 'true' defaults to base64.
The reason being that base64 encoding is the most common way to preserve a value with new lines, or a json value, which finds me constantly calling things *_B64 so that I remember they need to be Buffer.from and toString('ascii') in the code. Before my suffix people forgot, a lot.
@antony how much of an impediment would it be to require a validator for that? (validate can transform things arbitrarily, including decoding stuff) It feels slightly too opinionated to be a first class part of the API, but maybe it's more common than I'm imagining?
@Rich-Harris it's fine as a validator, and I can live with that, though perhaps feels a little shoehorn-ish. You could make the attribute only base64 or false, no reason to include a custom decoder really , but I see the technique a lot in docs now, I suppose as security gets more complex.
Nice! Just a suggestion, and this probably relates more to when the feature becomes official, but to configure it, instead of importing an arbitrary module, wouldn't it make more sense to make it available through kit.env ?
Currently this option is an object that defines how SvelteKit searches for variables (env file location and variables' names' prefix). Maybe it can be replaced by something like :
/** @type {import("@sveltejs/kit").Config} */
const config = {
// ...
kit: {
// ...
env: CONFIGURATION;
}
// ...
};
Where CONFIGURATION could be :
- a small definition if only a few variables are needed :
defineEnvVars({ POSTGRES_URL: {}, /* ... */})
- an imported object from another module :
// src/env.ts
import { defineEnvVars } from "@sveltejs/kit/hooks";
export default defineEnvVars({ /* ...lots of variables... */})
// svelte.config.js
import env from "./src/env";
const config = {
// ...
kit: {
// ...
env
}
// ...
};
- or a default method made available by SvelteKit to enable the current behavior :
import { detectEnvVars } from "@sveltejs/kit/hooks"
const config = {
// ...
kit: {
// ...
// Pass it the currently available options
env: detectEnvVars({ dir, publicPrefix, privatePrefix });
}
// ...
};
This may mean having to force users to pick one, though the default generated svelte.config.js could define it by default using one of the two methods made available. However this has the advantage to make it more of a conscious decision about how are environment variables managed.
It has to be a separate module, otherwise your app would need to import svelte.config.js in order to access the validators at runtime - that, in turn, would import @sveltejs/kit/vite and every other bit of junk in there
It has to be a separate module, otherwise your app would need to import
svelte.config.jsin order to access the validators at runtime - that, in turn, would import@sveltejs/kit/viteand every other bit of junk in there
You're right, I hadn't thought of that! Then i guess you could scratch option 1 but would 2 & 3 be feasible ?
Maybe 2 could instead be a string path to the module (e.g. src/env.ts) and 3 could kind of stay the same as currently, but replace it at run/build by a default generated module whose behavior is to import process.env (or equivalent if dir is defined) by following the related configuration (dir, publicPrefix, privatePrefix) with basically something like :
// generated-env-module.js
// inserted by reading config.kit.env
const envOptions = {
dir: "...",
publicPrefix: "...",
privatePrefix: "...",
};
export const variables = defineEnvVars(
Object.fromEntries(
Object.entries(process.env).map(([variable, value]) => [
variable,
{ public: variable.startsWith(envOptions.publicPrefix), static: "..." },
]),
),
);
Sorry I'm not sure if that's optimal in terms of implementation, I'm mainly talking about how it would affect the end user when they configure their svelte.config.js. I worry that loading an arbitrary module whose path isn't present in the config may be confusing (e.g. debugging a codebase you haven't created and not understanding why variables are loaded).
Note that this may be overkill or undesirable if you intend to make it the default for users to have to explicitly define the variables they use.
Option 2 is basically what we have already, except that instead of giving people enough rope to hang themselves flexibility over where env vars are defined, they're in a predictable, conventional location.
Option 3 sounds like it relies on static analysis trickery to extract things from the config and generate a module from them? That won't work at scale.
Some additional context: we have a plan to make svelte.config.js itself unnecessary, by allowing you to specify config directly in vite.config.ts. (It turns out that there are limits to how far you can bend Vite to your will if config is split between two places, and we're in the process of hitting those limits, so it makes sense to unify them.) Extracting stuff from the Vite config would be even harder.
Hi! I would change validate to schema (or maybe ), as validate naively implies being a (val: string) => boolean predicate
Ooh yeah I like schema
Really nice addition to the framework! I'm in the process of integrating Varlock (https://varlock.dev) in a big project at work, but I much prefer this approach. The only thing missing for me would be a way to load the actual values of the variables. For example, I might store all my variables in 1password and use the 1p cli to pull those values and populate the env. That's what drew me to Varlock in the first place. What do you think? Would that be out of scope? I know it's not a small ask, but it would be incredible to have that natively in Sveltekit.
Can you elaborate on what that would look like? Isn't it just this? (I haven't used the 1Password CLI so this is just me guessing from the docs)
op run --environment xyz123 -- pnpm devI have a question here. AFAIK (I might be wrong here), static private variables are not currently inlined, due to security reasons. Are they still allowed in this PR? I feel like they should be disallowed if they are not inlined, and SveleKit should emit a warning/disclaimer/require opt-in if they are.
Static private vars are inlined, in your server code (never in your client code). In today's world you opt in by importing the variable from $env/static/private and using it, with this PR you opt in by a) defining the variable as static: true (default is false) and importing/using it.
Yep sorry was on my phone. So what I'm thinking is we could be able to populate env vars dynamically from this config.
import { defineEnvVars } from '@sveltejs/kit/hooks';
import * as v from 'valibot';
import { execSync } from 'child_process';
export const variables = defineEnvVars({
POSTGRES_URL: {
// Could be a static value
value: 'postgresql://...',
// Can be derived from a main env variable (maybe Vite's `mode`?)
value: ({ mode }) => mode === 'production' ? 'postgresql://...' : 'local-db-url'
// The function can be async, so we can fetch the value from somewhere else
value: async () => vercel.environment.getSharedEnvVar({
id: "<id>",
teamId: "team_1a2b3c4d5e6f7g8h9i0j1k2l",
slug: "my-team-url-slug",
});
// The 1Password CLI could be used like this, but it could be any CLI
value: ({ mode }) => execSync(`op read op://${mode}/postgresurl`)
},
});
Doing this for every value is fine, but there could also be a way to load an env file in one go. Maybe something like:
// This function returns a string, formatted like the content of an env file (e.g. `MY_VAR=value`)
export const loadEnv = ({ mode }) => execSync(`op environment read ${mode}`);
This is a very rough idea, but thats a really cool feature of Varlock, allowing us to dispense with the multiple local .env files and have all our secrets in one secure location.
Hi! I would change
validatetoschema(or maybe ), asvalidatenaively implies being a(val: string) => booleanpredicate
Ah yes - this solves my concerns around validate parsing/coercing too. Nice :)
@f-elix still not completely sure I understand — for the Vercel example, if you need env vars locally you don't need to manage anything yourself you can just do vc env pull, and if building/running on the platform they're just... there, so using vercel.environment.getSharedEnvVar (is that a real API? not familiar) seems unnecessary.
For the 1Password example, what's the advantage of calling the CLI from inside src/env.ts rather than in the pnpm dev/pnpm build command?
As for loadEnv, you'd miss out on the ability to configure the variable with public, static, etc.
Having said all that: if you do need to load variables directly inside src/env.ts then schema already gives you that flexibility. The one missing piece is that we currently disallow async schemas, but I actually don't think there'd be any problem allowing them:
import { defineEnvVars } from '@sveltejs/kit/hooks';
import * as v from 'valibot';
import { execSync } from 'child_process';
const mode = import.meta.env.MODE;
const value = (v: string | Promise<string>) => v.pipeAsync(v.unknown(), v.transformAsync(() => v));
export const variables = defineEnvVars({
POSTGRES_URL: {
// Could be a static value
schema: value('postgresql://...'),
// Can be derived from import.meta.env.MODE
schema: value(mode === 'production' ? 'postgresql://...' : 'local-db-url')
// The value can be async, so we can fetch the value from somewhere else
schema: value(vercel.environment.getSharedEnvVar({
id: "<id>",
teamId: "team_1a2b3c4d5e6f7g8h9i0j1k2l",
slug: "my-team-url-slug",
}));
// The 1Password CLI could be used like this, but it could be any CLI
schema: value(execSync(`op read op://${mode}/postgresurl`))
},
});
We should probably also expose $app/env (and $lib etc) so that you can import dev, making import.meta.env.MODE unnecessary.
It's also straightforward to configure (or, if desired) load things in bulk if necessary:
import { defineEnvVars } from '@sveltejs/kit/hooks';
import { systemEnvVars } from '@sveltejs/adapter-vercel/env'; // hypothetical!
import { load1PasswordEnvironment } from '$lib/server/1password';
import { dev } from '$app/env';
export const variables = defineEnvVars({
...systemEnvVars(), // VERCEL_URL etc
...load1PasswordEnvironment(dev ? 'development' : 'production'),
MY_OTHER_STUFF: {...}
});Actually I take it back, we can't allow async validators right now as that would force generateEnvModule to be async which would be a breaking change (albeit a relative minor one). Should be possible in v3 though.
In the meantime you can of course await directly in src/env:
schema: value(vercel.environment.getSharedEnvVar({
schema: value(await vercel.environment.getSharedEnvVar({Oh I see. I only saw schema as way to validate the value, not populate the variable, but you're right. Its actually much better.
To answer your question regarding calling a CLI or an SDK from env.ts (doesn't have to be Vercel, could be anything), its imo a cleaner and more flexible way than to hardcode that in a package.json script. You could have multiple environments (production, staging, tests, local dev, etc.) with different values, and that would become very messy to handle in package.json. In any case, I think schema gives me the flexibility I need for this.
I also wasn't clear with my loadEnv idea. That function would live next to defineEnvVars, its not meant to replace it. I saw defineEnvVars as a way to define a schema, and loadEnv as an alternative way to provide the values without needing local .env files.
So you can forget my first suggestion (the value property), schema already takes care of that. As for loadEnv, it would be a convenience function. I could for sure create a module, fetch my env vars, transform that into the defineEnvVars input and spread it like in your example, but it'd be nice if I didn't have to :).
Does that make more sense to you?
What would loadEnv do exactly?
I think I explained this poorly.
What I had in mind is a function that returns dotenv file contents as a string, e.g. FOO=bar\nPUBLIC_X=y. SvelteKit would then parse that exactly like an .env file, validate it against the defineEnvVars schema and feed the resulting values into the $env/* modules.
That loader determines where the dotenv contents come from. The schema remains the source of truth for what variables exist, whether they are public/private/static, docs, validation, etc.
The reason this seems useful to me is that it keeps src/env.ts as the central place where env is declared, while still allowing the raw values to come from userland tooling rather than package scripts or shared env files.
I’m not sure whether this belongs in core, especially since there are details around precedence, async loading, build-time vs runtime behavior, and whether it should mutate process.env. But that was the shape I meant. loadEnv is probably also the wrong name — it would be more like defineEnvSource, loadDotenv, or something along those lines.
Ah, I think I understand — so basically this, without the pointlessly hard-coded string?
export const variables = defineEnvVars({
FOO: {...},
BAR: {...}
});
export const load = async () => `
FOO=123
BAR=456
`;
I think it's a candidate for a future addition, though probably not part of this PR
Yes exactly! But yeah agreed that its out of scope for this PR. Really looking forward to this one :)
Great! It works with zod4 too.
Pro tip: You can prefix GitHub URLs of issues, PRs or discussions with svcl.dev/ to view them on this page! Also try it on a GitHub release ;)