Feb 28, 2024

Hiding pages from sitemap in Astro Content

Astro is a modern static site generator that brings together the best of traditional static site generators and modern build tools. When working with content-driven websites, there might be cases where you want to hide certain pages from public access. In this blog post, we’ll explore a TypeScript code snippet that accomplishes this task using Astro.

Understanding the Code

Let’s delve into the TypeScript code responsible for filtering out non-public or standalone pages.

typescript
1import fs from 'fs';
2import yaml from 'js-yaml';
3import path from 'path';

4interface FileData {
5  slug: string;
6  public: boolean;
7  standalone: boolean;
8}

9const directory = path.join(process.cwd(), 'src', 'content');
10const collectionName = 'notes';

11export function getFilteredSlugs(url: string): string[] {
12  const slugs: string[] = [];

13  function traverseDirectory(dir: string) {
14    const files = fs.readdirSync(dir);

15    files.forEach((file) => {
16      const filePath = path.join(dir, file);
17      const stat = fs.statSync(filePath);

18      if (stat.isDirectory()) {
19        traverseDirectory(filePath);
20      } else if (path.extname(file) === '.md' || path.extname(file) === '.mdx') {
21        const fileContent = fs.readFileSync(filePath, 'utf8');
22        const yamlData = fileContent.split('---')[1];
23        const fileData: FileData = yaml.load(yamlData) as FileData;
24        const fileNameSlug = file.replace(/\.mdx?$/, '').replace(/ /g, '-').toLowerCase().replace(/[^a-z0-9-]/g, '');
25        const preSlugPath = path.dirname(filePath).replace(directory, '').replace(collectionName, '')
26        const slug = fileData.slug ? `/${fileData.slug}` : path.join(preSlugPath, fileNameSlug);

27        if (!fileData.public || fileData.standalone) {
28          slugs.push(`${url}${slug}/`);
29        }
30      }
31    });
32  }

33  traverseDirectory(directory);

34  return slugs;
35}

Code Breakdown

  1. FileData Interface: Describes the structure of the YAML front matter in each markdown file, including the slug, public, and standalone properties.

  2. Configuration Variables: Defines the root directory and collection name for content files.

  3. getFilteredSlugs Function: This function takes a base URL as a parameter and returns an array of filtered slugs.

  4. traverseDirectory Function: Recursively traverses the content directory, extracting information from markdown files.

  5. YAML Parsing: Reads the YAML front matter of each markdown file to obtain metadata such as slug, public, and standalone.

  6. Slug Generation: Generates slugs based on file content and YAML data, considering the specified rules for replacement and transformation.

  7. Filtering: Adds slugs to the result array only if the page is non-public or standalone.

Implementation in Astro

To use this code in an Astro project, import the getFilteredSlugs function and call it with the base URL. The returned array of slugs can then be utilized in your Astro configuration to control the visibility of pages.

typescript
1import { getFilteredSlugs } from './src/utils/sitemap.util.ts';

2const site = 'https://notes.wesamjabali.com'

3const filteredSlugs = getFilteredSlugs(site);

4export default defineConfig({
5  site,
6  integrations: [mdx(), sitemap({ filter: (pageUrl) => !sitemapIgnoreUrls.includes(pageUrl) })],
7  image: {
8    service: squooshImageService()
9  }
10});

With this setup, you can easily manage the visibility of pages in your Astro-generated site based on the specified criteria in the YAML front matter.

In conclusion, this TypeScript code offers a flexible solution for hiding non-public or standalone pages in an Astro content-driven website, giving you control over which pages are accessible to the public.