Build-Time Data

Lakea uses build-time data fetching for small datasets from APIs with CORS restrictions or slow response times. This guide explains the pattern.

When to Use

Use build-time data fetching when:

• Dataset is small (under 10MB)
• API has CORS restrictions (can’t call from browser)
• API is slow or rate-limited
• Data doesn’t change frequently (daily/weekly updates are fine)

Use client-side fetching when:

• Data changes frequently
• User-specific queries needed
• Dataset is too large to bundle

Pattern Overview

1. Create a fetch script in tools/
2. Script runs in Node.js (no CORS)
3. Output saved to apps/{app}/public/data/
4. App loads the static JSON at runtime

Creating a Fetch Script

File Location

tools/fetch-{source}-data.ts

Template

#!/usr/bin/env npx tsx
/**
 * Fetch {Source} data at build time
 *
 * Usage: pnpm fetch:{source}
 */

import { writeFileSync, mkdirSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';

const __dirname = dirname(fileURLToPath(import.meta.url));
const OUTPUT_DIR = join(__dirname, '../apps/{app}/public/data');
const OUTPUT_FILE = join(OUTPUT_DIR, '{filename}.json');

async function fetchData() {
  console.log('Fetching from {Source}...');
  const response = await fetch('{API_URL}');

  if (!response.ok) {
    throw new Error(`API error: ${response.status}`);
  }

  return response.json();
}

async function main() {
  try {
    const data = await fetchData();
    console.log(`Fetched ${data.length} items`);

    mkdirSync(OUTPUT_DIR, { recursive: true });

    const output = {
      fetchedAt: new Date().toISOString(),
      source: '{Source Name}',
      sourceUrl: '{Source URL}',
      count: data.length,
      data,
    };

    writeFileSync(OUTPUT_FILE, JSON.stringify(output, null, 2));
    console.log(`Saved to ${OUTPUT_FILE}`);
  } catch (error) {
    console.error('Failed:', error);
    process.exit(1);
  }
}

main();

Key Points

• Shebang: #!/usr/bin/env npx tsx makes script directly executable
• Output format: Wrap data with metadata (fetchedAt, source, count)
• Error handling: Exit with code 1 on failure for CI detection

Adding to Package Scripts

In package.json:

{
  "scripts": {
    "fetch:nasa": "npx tsx tools/fetch-nasa-data.ts"
  }
}

Predev Hook

Ensure data exists before development:

{
  "scripts": {
    "predev": "[ -f apps/{app}/public/data/{file}.json ] || pnpm fetch:{source}"
  }
}

This checks if the file exists and fetches only if missing.

Consuming in App

Load the static JSON in your React app:

const [data, setData] = useState(null);
const [loading, setLoading] = useState(true);

useEffect(() => {
  fetch('/data/exoplanets.json')
    .then((res) => res.json())
    .then((result) => {
      setData(result.data);
    })
    .finally(() => {
      setLoading(false);
    });
}, []);

Example: NASA Exoplanet Archive

The exoplanet-catalog app uses this pattern:

Component	Path
Script	tools/fetch-nasa-data.ts
Output	apps/exoplanet-catalog/public/data/exoplanets.json
Command	pnpm fetch:nasa
Consumer	apps/exoplanet-catalog/src/App.tsx

The script:

1. Queries NASA TAP API with ADQL
2. Fetches all confirmed exoplanets (~5,700)
3. Saves with metadata to static JSON
4. App loads from /data/exoplanets.json

Production Builds

For Cloudflare Pages, include the fetch in build command:

pnpm fetch:nasa && pnpm nx build @lakea/exoplanet-catalog

See Deployment for full setup.