research the best ways to set convex env vars in a local deployment running in a sandbox. i want to ...

Use Convex CLI or Dashboard for Dev Environment Variables

In Convex, backend environment variables are managed per deployment. For a sandbox/dev deployment, the recommended approach is to use the Convex CLI or the Convex dashboard. You can set variables from your terminal with commands like: npx convex env set YOUR_VAR "your-value" (docs.convex.dev). By default, this targets your dev (sandbox) environment. You can verify them with npx convex env list (docs.convex.dev) or see/edit them in the Convex web dashboard. This ensures the variables (like API keys) are stored in Convex’s backend and accessible via process.env.YOUR_VAR in your Convex functions (docs.convex.dev). If your dev server was failing due to missing env vars, make sure you’ve added those keys using the CLI or dashboard before running your code. Convex will load those values into the backend runtime for you.

Load a Local Env File for Development

When running Convex in a local development mode, you should ensure the required env vars are present in the local environment. The Convex CLI (npx convex dev) uses a file named .env.local in your project directory to configure local deployments (deepwiki.com). In fact, the CLI will create/update this file automatically with some keys (like CONVEX_DEPLOYMENT and the dev URL) and add it to .gitignore. A good practice is to put your custom env vars in that .env.local file as well. For example, you might add lines for your API keys or secrets there. Convex’s docs suggest placing needed variables in .env.local so that npx convex dev picks them up automatically (stack.convex.dev). The next time you start the dev server, the CLI will read .env.local and inject those values so that your Convex backend has them. (If you previously tried using a .env file, note that Convex specifically looks for .env.local by default, depending on your framework.)

Alternatively, you can export your .env file contents into the shell before launching Convex. For instance, you could run export $(grep -v '^#' .env | xargs) && npx convex dev to load the variables into your environment. This way, when the Convex dev server starts, it inherits those env vars from the parent shell. However, be aware that the Convex CLI will still create a .env.local on startup if one doesn’t exist (github.com). Overall, using .env.local is the simplest method since Convex is designed to utilize it for local config.

Ensure Env Vars for a Self-Hosted Local Backend (if applicable)

If by “local deployment in a sandbox” you mean you’re self-hosting the Convex backend (running the Convex server on localhost or Docker instead of Convex cloud), then you must pass the needed env vars to that local server process. In the self-hosted setup, certain Convex config variables (like CONVEX_CLOUD_ORIGIN and CONVEX_SITE_ORIGIN) may need to be set for the backend to run properly (stack.convex.dev). The Convex docs note that these can be specified in your Docker Compose or as environment variables when launching the Convex container (stack.convex.dev). Similarly, if your Convex functions rely on secrets (API keys, etc.), you’d supply those as env vars to the local server. You can do this by defining them in your shell or compose file, or by using a tool like dotenv in the startup script for the Convex server. For example, in a Docker Compose file you might include an env_file: .env or list the variables under an environment: section so the Convex backend container reads them on startup. If you’re running the Convex binary directly, you can source your .env and then run the binary so it has the vars in its environment. Without these values, a self-hosted Convex backend might fail to start or your functions will throw errors when accessing process.env. Always double-check that the env vars are loaded by printing them or checking logs at startup.

Summary

In short, the best practice is: for Convex cloud deployments (including your personal dev sandbox), use the Convex CLI or dashboard to set environment variables (so they live in Convex’s environment store). For local development and testing, keep those variables in a .env.local file (or export them in your shell) so that convex dev runs with all needed values. This way your backend won’t crash due to missing secrets. Following Convex’s standard setup for env vars – CLI for cloud and .env.local for local – will ensure your dev server runs smoothly with the correct configuration. (stack.convex.dev) (docs.convex.dev)

Configuring Environment Variables in Local Convex Deployments: A Comprehensive Guide

This report explores the practical challenges and solutions for managing environment variables when running Convex in a local sandbox deployment. The core issue involves a circular dependency: local deployments require environment variables to function, yet accessing those variables requires the deployment to be running. This research examines multiple approaches to resolve this fundamental tension and establish a reliable workflow for local development environments.

Understanding the Local Deployment Environment and Configuration Architecture

Local Convex deployments operate fundamentally differently from cloud-hosted deployments in how they manage configuration and state. When you run npx convex dev --local, the Convex backend runs as a subprocess on your local machine, storing state in the ~/.convex/ directory rather than relying on cloud infrastructure. This architectural decision provides significant benefits for development, as it eliminates upload bandwidth consumption from your quota and accelerates code synchronization cycles.

However, this architectural approach creates distinct configuration challenges because there are actually two separate layers of environment variable management in Convex projects. The first layer consists of project-level configuration variables that control how the Convex CLI behaves and which deployment to target, stored in .env.local or .env files in your project root[4][4]. The second layer comprises deployment-level environment variables that your Convex functions access through process.env at runtime, which are managed per-deployment and stored independently from project configuration files[1][1][1][1][1][1].

The confusion stems from the fact that these two layers are entirely independent systems[1]. When you run npx convex dev, the CLI reads from .env.local to discover which deployment to work with via the CONVEX_DEPLOYMENT variable, but this is different from the environment variables that will be available to your functions through process.env inside your functions[1][4][4]. For local deployments specifically, the state and configuration are stored in ~/.convex/convex-backend-state/ directories, and the backend process needs to start successfully before you can interact with it.

The circular problem you're experiencing arises because many developers intuitively expect that setting variables in .env.local or through shell environment variables before running npx convex dev --local will automatically populate the deployment's environment variables. In reality, .env.local is purely for CLI configuration and frontend application configuration (prefixed with NEXT_PUBLIC_, REACT_APP_, or VITE_ depending on your framework), not for your backend functions[4][4][4].

The Architecture of Environment Variables: Distinguishing Between Deployment-Level and Project-Level Configuration

To properly understand and solve your problem, you must recognize that Convex maintains strict separation between three distinct scopes of environment variables, each serving a different purpose in the development workflow[1][4][4][1][1][1][1]. Understanding these distinctions is critical because confusing them is the root cause of the circular dependency issue.

Project-level configuration variables are stored in .env.local and control how the Convex CLI operates. These include CONVEX_DEPLOYMENT, which tells the CLI which deployment to target for npx convex dev and npx convex deploy commands[4][4]. These variables are never visible to your Convex functions[1][1][1][1]. When you run npx convex dev, the CLI reads .env.local and uses this information to determine which deployment to synchronize code to. For local deployments, npx convex dev --local doesn't even require a .env.local file initially, though one will be created with the CONVEX_DEPLOYMENT variable pointing to your local backend.

Frontend application configuration variables are separate again and are accessed by your frontend code before it connects to Convex. In Next.js, these must be prefixed with NEXT_PUBLIC_ to be bundled with your frontend. In Vite projects, they must start with VITE_. These frontend variables are typically stored in .env, .env.local, or .env.production and define things like the Convex deployment URL your frontend should connect to. These variables are evaluated at build time and bundled into your frontend application.

Deployment-level environment variables are the ones your Convex backend functions access through process.env. These are completely separate from .env.local and .env files. They are managed per-deployment through the Convex dashboard under "Deployment Settings" or via the CLI with npx convex env set, npx convex env list, and npx convex env get commands[1][1][1][1][1][1][1]. These variables are not evaluated until your functions execute, and they can be different for each deployment (dev and prod can have different values for the same variable name)[1][1][1][1][1][1][1].

This three-layer architecture explains why your local deployment fails to start when you expect it to pick up environment variables from a .env.local file. The backend cannot access your project's .env.local file because that file is not designed for deployment-level configuration[1][4][4][1][1][1][1]. Instead, you must set deployment-level environment variables through explicit mechanisms that bypass the .env.local system entirely.

Solution Strategy One: Pre-Initialization with Deployment Environment Variables Before Backend Startup

The most straightforward approach to resolve your circular dependency problem is to recognize that deployment environment variables must be set before you expect your functions to use them, but they can be set while the local backend is already running[1][1][1][1][1][1]. The key insight is that you don't need environment variables to exist before the backend starts—you need them to exist before your functions execute, which gives you a window during which the backend is running but your functions haven't been called yet.

To implement this approach, start your local deployment with npx convex dev --local without worrying about environment variables in the first place. The backend will start successfully because it doesn't require deployment environment variables to initialize. Once the backend is running, you can then set your deployment environment variables using the Convex CLI[1][1][1][1][1][1]. The command structure is straightforward:

npx convex env set VARIABLE_NAME variable_value

When running against a local deployment, this command automatically targets your local backend. You can set multiple environment variables in sequence:

npx convex env set DATABASE_URL "postgres://localhost:5432/mydb"
npx convex env set API_KEY "secret-key-value"
npx convex env set ENABLE_FEATURE true

After setting your environment variables via npx convex env set, you can verify they were applied correctly with:

npx convex env list

This two-phase workflow—first start the backend, then configure environment variables—is the foundation for solving your problem[1][1][1][1][1][1]. The advantage of this approach is simplicity and transparency. You see exactly what environment variables exist and understand when they were set.

However, this manual approach has limitations for automation. If you're trying to run your local deployment in a sandbox environment that needs to be fully configured and ready-to-use automatically, performing these steps manually becomes impractical. This is where automation strategies become necessary.

Solution Strategy Two: Automated Environment Variable Injection with a Startup Script

To automate environment variable setup for your local deployment, create a startup script that orchestrates the initialization sequence. This script should handle the sequential nature of local backend startup and environment variable configuration, ensuring both operations complete successfully before your application attempts to use the environment variables[1][1][1][1][1][1].

Create a file called setup-local-env.sh in your project root:

#!/bin/bash
set -e

# Start the local Convex backend in the background
echo "Starting local Convex backend..."
npx convex dev --local > /tmp/convex-backend.log 2>&1 &
BACKEND_PID=$!

# Wait for the backend to be ready
echo "Waiting for backend to start..."
sleep 5

# Check if backend is running
if ! kill -0 $BACKEND_PID 2>/dev/null; then
  echo "Backend failed to start. Check /tmp/convex-backend.log for details."
  exit 1
fi

# Now set environment variables from your .env.local file
echo "Setting deployment environment variables..."

# Read your .env file and extract variables
while IFS='=' read -r key value; do
  # Skip empty lines and comments
  [[ -z "$key" || "$key" =~ ^[[:space:]]*# ]] && continue
  
  # Skip Convex CLI configuration variables
  [[ "$key" == "CONVEX_DEPLOYMENT" || "$key" == "CONVEX_CLOUD_URL" ]] && continue
  
  # Set the environment variable in your deployment
  npx convex env set "$key" "$value"
  echo "Set $key"
done < .env.deployment

echo "Local Convex setup complete!"
echo "Backend process ID: $BACKEND_PID"
echo "To stop the backend, run: kill $BACKEND_PID"

# Keep the script running to maintain the backend
wait $BACKEND_PID

This script demonstrates several important patterns for managing local deployments with environment variables. First, it starts the Convex backend as a background process using npx convex dev --local. Second, it waits a brief period to allow the backend to initialize fully. Third, it reads environment variables from a dedicated .env.deployment file (separate from .env.local) and uses npx convex env set to configure them[1][1][1][1][1][1]. Finally, it maintains the backend process by waiting on its PID, ensuring that when you terminate the script, the backend also stops.

The key distinguishing feature is the use of a separate .env.deployment file for deployment-level variables rather than trying to read from .env.local. This separation is crucial because it makes the intent explicit: variables in .env.deployment are for your Convex backend functions, while .env.local contains CLI configuration and frontend variables[4][4][4].

Your .env.deployment file would look like:

DATABASE_URL=postgres://localhost:5432/mydb
API_KEY=dev-secret-key
SENDGRID_API_KEY=dev-sendgrid-key
JWT_SECRET=dev-jwt-secret

To use this automated setup in your sandbox environment, run:

chmod +x setup-local-env.sh
./setup-local-env.sh

The backend will start and remain running with all environment variables configured. This approach is particularly useful in containerized or CI environments where you need deterministic, reproducible setup.

Solution Strategy Three: Using the Convex MCP Server for Programmatic Environment Variable Management

For more sophisticated automation scenarios, particularly when you need to set environment variables programmatically from other tools or scripts, the Convex Model Context Protocol (MCP) server provides a powerful abstraction layer[2][2]. The MCP server allows you to interact with your Convex deployment through standardized protocol tools, including complete environment variable management capabilities[2][2].

To set up the MCP server for local development, start it with:

npx -y convex@latest mcp start --local

This enables AI agents or other tools to interact with your local deployment through the MCP protocol[2][2]. The MCP server exposes environment variable tools including envSet, envGet, envList, and envRemove[2][2]. These tools can be invoked programmatically to manage your deployment's environment variables[2][2].

Once the MCP server is running, other applications can invoke these tools to set environment variables without needing to directly execute CLI commands. For example, an initialization script written in another language can communicate with the MCP server via stdio and request that specific environment variables be set[2][2].

The advantage of this approach is decoupling: your initialization logic doesn't need to directly invoke the Convex CLI, and it doesn't need to wait for specific delays between operations. The MCP server handles the details of communicating with your local backend[2][2]. This makes it easier to integrate Convex environment variable management into complex setup pipelines, build systems, or CI/CD workflows.

However, the MCP server approach requires additional setup and tooling. It's most beneficial when you already have systems in place that understand the MCP protocol, or when you need to manage Convex deployments as part of a larger automation framework[2][2].

Solution Strategy Four: Conditional Function Implementation Based on Environment Variables

While not a direct solution to the environment variable loading problem, an important defensive programming practice is to structure your Convex functions to gracefully handle missing environment variables[9][10]. This approach acknowledges that some environment variables might not always be present during development or testing.

When accessing environment variables in your Convex functions, always check whether they exist before using them:

import { query } from "./_generated/server";

export const sendNotification = query({
  args: {},
  handler: async (ctx) => {
    const sendgridKey = process.env.SENDGRID_API_KEY;
    
    // Handle missing environment variable gracefully
    if (!sendgridKey) {
      console.warn("SENDGRID_API_KEY not configured");
      return { status: "skipped", reason: "API key not configured" };
    }
    
    // Proceed with using the API key
    return { status: "sent" };
  },
});

This pattern is particularly important for optional integrations where you want your application to function with reduced capabilities rather than failing entirely when optional environment variables aren't set[9][10]. A critical caveat: do not condition function exports on environment variables[1][1][1][1][1][1][1]. This means you should never write code like:

// THIS WILL NOT WORK - DO NOT DO THIS
export const myFunc = process.env.DEBUG ? mutation(...) : internalMutation(...);

This pattern fails because Convex determines which functions are exported at deployment time when your code is pushed, not at runtime when the function is called[1][1][1][1][1][1][1]. If you change the DEBUG environment variable after deployment, your function definition won't change, causing runtime errors[1][1][1][1][1][1][1].

Solution Strategy Five: Local Deployment with Self-Hosting Architecture

For the most control over your local deployment environment, particularly in advanced sandbox scenarios, consider using the open-source convex-backend with self-hosting. This approach gives you complete control over initialization, configuration, and environment variable handling.

When self-hosting, you can use Docker Compose to manage both your backend and any supporting services:

npx degit get-convex/convex-backend/self-hosted/docker/docker-compose.yml docker-compose.yml
docker compose pull
docker compose up

Create a .env file in the same directory as your docker-compose.yml:

CONVEX_CLOUD_ORIGIN=http://localhost:3210
CONVEX_SITE_ORIGIN=http://localhost:3211
NEXT_PUBLIC_DEPLOYMENT_URL=http://localhost:3210

These variables configure how the self-hosted backend exposes itself to your applications. Unlike deployment-level variables, these are infrastructure configuration variables that affect how clients communicate with your backend.

Once your self-hosted backend is running, you can set deployment environment variables using the generated admin key:

CONVEX_SELF_HOSTED_URL='http://localhost:3210' \
CONVEX_SELF_HOSTED_ADMIN_KEY='<your-admin-key>' \
npx convex env set MY_VARIABLE my_value

The self-hosting approach is more complex to set up initially but provides the greatest flexibility for advanced sandbox scenarios where you need complete control over the entire infrastructure stack.

Handling Multi-Line Environment Variables and Special Cases

One particularly challenging scenario when setting environment variables is handling values that span multiple lines, such as PEM-formatted private keys for JWT authentication[8]. The standard npx convex env set command has limitations with multi-line values[8].

A practical workaround, as documented in the community, involves replacing newlines with spaces when setting the environment variable, then parsing it back in your code:

# Generate and store a key with spaces instead of newlines
JWT_PRIVATE_KEY=$(cat private.pem | tr '\n' ' ')
npx convex env set JWT_PRIVATE_KEY "$JWT_PRIVATE_KEY"

Then in your code, convert the spaces back to newlines:

const privateKey = process.env.JWT_PRIVATE_KEY?.replace(/ /g, '\n');

Alternatively, the Convex team has indicated interest in adding a --from-file flag to the convex env set command to make this use case easier, though as of the knowledge cutoff this feature was not yet implemented[8].

For now, the most reliable approach for multi-line values is to Base64-encode them before setting them as environment variables:

JWT_PRIVATE_KEY_B64=$(cat private.pem | base64)
npx convex env set JWT_PRIVATE_KEY_B64 "$JWT_PRIVATE_KEY_B64"

Then decode in your code:

const privateKeyB64 = process.env.JWT_PRIVATE_KEY_B64;
const privateKey = Buffer.from(privateKeyB64 || '', 'base64').toString('utf-8');

System Environment Variables and Reserved Variables in Local Deployments

Convex provides two system environment variables that are always available in your functions and should not be manually configured[1][1][1][1][1][1][1]. Understanding these prevents configuration mistakes:

CONVEX_CLOUD_URL is automatically set to your deployment's URL and is used by Convex clients to connect to your backend[1][1][1][1][1][1][1]. For cloud deployments, this is automatically set to something like https://dusty-nightingale-847.convex.cloud[1][1][1][1][1][1][1]. For local deployments, this defaults to http://localhost:3210.

CONVEX_SITE_URL is the URL where HTTP actions are served and is used for webhooks and external services[1][1][1][1][1][1][1]. For cloud deployments this might be https://dusty-nightingale-847.convex.site, and for local deployments typically http://localhost:3211.

These variables are automatically configured based on your deployment type and should never be manually set in .env.local or through npx convex env set[1][1][1][1][1][1][1].

Troubleshooting Common Local Deployment Startup Failures

When your local backend fails to start, particularly in sandbox environments, several environment-related issues can be the culprit[7][7][7][7]. One documented issue involves the TZ timezone environment variable interfering with local backend initialization[7][7]. If you encounter timeout errors when starting your local backend, try explicitly setting the timezone:

TZ=UTC npx convex dev --local

This simple fix has resolved startup failures in multiple reported cases[7][7]. Another approach when the local backend fails to start is to clear the cached state and reinitialize:

rm -rf ~/.convex/convex-backend-state/
npx convex dev --local

This removes all locally cached data and forces a clean initialization[7][7][7][7]. If your deployment configuration is corrupted, you may need to also reset your .env.local file and recreate your deployment configuration[7][7][7][7].

Best Practices for Environment Variables in Local Sandbox Deployments

Based on the patterns and issues documented in the Convex ecosystem, several best practices emerge for managing environment variables in local deployments:

Separate configuration concerns explicitly. Create distinct files for different types of configuration: .env.local for CLI configuration and frontend variables, .env.deployment for backend function environment variables, and shell environment variables only when necessary[4][4][4][1].

Automate the initialization sequence. Never rely on manual environment variable configuration in sandbox environments. Use startup scripts or automation frameworks to ensure consistent, reproducible setup[1][1][1][1][1][1].

Document environment variable requirements. Create a .env.deployment.example file that shows which environment variables are required and expected values:

DATABASE_URL=postgres://localhost:5432/mydb
API_KEY=your-dev-api-key
SENDGRID_API_KEY=your-sendgrid-dev-key

Validate environment variables at startup. In your initialization scripts, verify that required environment variables are actually set before proceeding:

required_vars=("DATABASE_URL" "API_KEY")
for var in "${required_vars[@]}"; do
  if ! npx convex env get "$var" > /dev/null 2>&1; then
    echo "Error: Required environment variable $var is not set"
    exit 1
  fi
done

Use different environment variable values for dev and prod. Take advantage of Convex's per-deployment environment variable system to use separate API keys, database URLs, and credentials for development versus production[1][1][1][1][1][1][1].

Conclusion

The circular dependency you're experiencing when trying to configure environment variables in a local Convex deployment stems from a fundamental misunderstanding about how Convex manages configuration across three distinct layers: CLI configuration (.env.local), frontend configuration (framework-prefixed variables), and deployment-level configuration (accessed via process.env in functions). These are entirely separate systems with different scoping rules and update mechanisms.

The most practical solution for your sandbox scenario is to implement an automated startup script that follows this sequence: first start the local backend with npx convex dev --local, second wait for initialization to complete, third set deployment environment variables using npx convex env set reading from a dedicated .env.deployment file, and finally signal readiness to the rest of your system. This approach resolves the circular dependency by recognizing that the backend doesn't need environment variables to initialize, only your functions need them before execution.

For more advanced scenarios, the Convex MCP server provides programmatic environment variable management, and self-hosting with convex-backend offers complete infrastructure control. Regardless of which approach you choose, the key principle remains consistent: separate your configuration concerns, automate your initialization sequence, and validate that required environment variables are actually set before attempting to use them.