DAG-FS Filesystem - Chapp-E Documentation

Overview

DAG-FS is Chapp-E's revolutionary filesystem that mimics how the human brain stores memories. Instead of storing files as discrete blocks on disk, DAG-FS stores "files" as distributed patterns of synaptic weights across neural networks.

🧠 Brain-Inspired: DAG-FS is based on neuroscience research showing that memories are stored as distributed patterns of synaptic strengths, not as discrete files in fixed locations. Every read operation strengthens the pattern (reconsolidation), just like accessing a memory in the brain.

✅ C-BASED IMPLEMENTATION: DAG-FS has been completely rewritten in C for Chappie OS. The filesystem is integrated directly into the kernel and initialized during kernel boot as part of the neural development sequence (Stage 4: DAG-FS Initialization).

🧬 KERNEL INTEGRATION: DAG-FS is not a separate module - it's part of the kernel itself. Just as the brain's memory system is part of the brain, DAG-FS is part of Chappie's kernel. Neurons have direct Ring-0 access to DAG-FS operations.

Traditional Filesystem vs. DAG-FS

Feature	Traditional Filesystem	DAG-FS (Brain Analogy)
Storage Unit	Block on disk	Synaptic Strength (Edge Weight)
Organization	Hierarchical folders/directories	Contextual Associations (Tags)
File Access	File path lookup (Read-only)	Pattern matching (Read/Write)
Data Integrity	Fixed location, checksummed	Dynamic, reconstructive (can change upon access)
Read Operation	Static data retrieval	Dynamic reconstruction + reconsolidation
Write Operation	Overwrite data	Hebbian learning (weight updates)
Delete Operation	Remove file	Long-Term Depression (LTD - weaken weights)

Core Concepts

1. Nodes (Neurons)

Nodes represent individual neurons in Chapp-E's neural network layers. Each node can participate in multiple "files" (patterns) simultaneously, just like real neurons participate in multiple memories.

Location: Neural network layers (16 input, 8 hidden, 4 output)
Function: Activation units that process information
Storage: Activation values, thresholds
Multi-pattern: Each neuron can be part of thousands of different patterns

2. Edges (Synapses)

Edges represent synaptic connections between neurons. The weight of each edge is the actual "data" - stronger connections mean stronger associations. These weights are stored in Chapp-E's neural network weight matrices.

Storage: Weight values (64-bit qwords)
Dynamic: Weights change with access (reconsolidation)
Directional: Information flows in specific directions
Hebbian: "Cells that fire together, wire together"

3. Patterns (Memories/Files)

Patterns are distributed "files" stored as specific configurations of weights across the network. A pattern only "exists" when it's activated - it's an emergent property of the network, not a fixed location.

Storage: Distributed across multiple nodes/edges
Access: Pattern matching → forward pass reconstruction
Emergent: Only exists when activated
Metadata: 64 bytes per pattern (ID, tags, weights pointer, strength, access count)
Capacity: Up to 1024 patterns supported

4. Tags (Context/Associations)

Tags are contextual cues that allow pattern matching. Instead of file paths like /home/user/file.txt, you query by tags like "first day of school" + "chalk" + "happy".

Storage: Metadata linking patterns to contexts
Query: Pattern-based access (not path-based)
Capacity: Up to 512 tags supported
Structure: 128 bytes per tag (ID, name, pattern list, count, strength)

Complete API Reference

Initialization

void dagfs_init(void);

Initialize the DAG-FS filesystem. Called automatically during kernel boot as part of the neural development sequence (Stage 4: DAG-FS Initialization). Clears pattern storage and index, initializes counters.

Kernel Integration: This function is called from neural_system_init() in system/kernel/core/neural_init.c, which is called from kernel_init() in system/kernel/core/init.c.

Pattern Operations

Create Pattern

pattern_id_t dagfs_create_pattern(const char *tags, const void *data, size_t length);

Description: Create a new pattern from data and tags. Stores data as weight configuration in neural network.

Parameters:

tags - Pointer to tag string (contextual cues)
data - Pointer to data to store
length - Length of data in bytes

Returns: Pattern ID on success, 0 on error

Read Pattern

void *dagfs_read_pattern(pattern_id_t pattern_id);

Description: Reconstruct pattern from stored weights. Strengthens pattern on access (reconsolidation).

Parameters:

pattern_id - Pattern ID to read

Returns: Pointer to reconstructed data, or NULL on error

Note: This operation modifies the pattern (strengthens weights) - just like accessing a memory in the brain!

Write Pattern

int dagfs_write_pattern(pattern_id_t pattern_id, const void *data, size_t length);

Description: Update pattern with new data using Hebbian learning (weight updates).

Parameters:

pattern_id - Pattern ID to update
data - Pointer to new data
length - Length of data in bytes

Returns: 0 on success, -1 on error

Delete Pattern

int dagfs_delete_pattern(pattern_id_t pattern_id);

Description: Delete pattern using Long-Term Depression (LTD) - weakens associated weights.

Parameters:

pattern_id - Pattern ID to delete

Returns: 0 on success, -1 on error

Note: Weights are weakened by ~10% each time, simulating forgetting.

Query Pattern

pattern_id_t dagfs_query(const char *pattern, size_t length);

Description: Query patterns by tag/context matching.

Parameters:

pattern - Query string/tags
length - Length of query string

Returns: Pattern ID if found, 0 if not found

Tag Operations

Add Tag

tag_id_t dagfs_add_tag(pattern_id_t pattern_id, const char *tag);

Description: Associate a tag with a pattern for contextual access.

Returns: Tag ID on success, 0 on error

Query by Tag

pattern_id_t dagfs_query_by_tag(const char *tag);

Description: Find patterns associated with a specific tag.

Returns: Pattern ID if found, 0 if not found

Pattern Completion

Complete Pattern

int dagfs_complete_pattern(pattern_id_t pattern_id, const void *partial_input,
                           size_t input_length, void *output, size_t output_size);

Description: Complete a pattern from partial input (like brain memory recall). Runs forward pass to reconstruct full pattern.

Parameters:

pattern_id - Pattern ID to complete
partial_input - Partial input data
input_length - Length of partial input
output - Buffer for completed pattern
output_size - Size of output buffer

Returns: Bytes written on success, -1 on error

Note: Strengthens pattern on completion (reconsolidation).

Associate Patterns

int dagfs_associate_patterns(pattern_id_t pattern1, pattern_id_t pattern2);

Description: Link two patterns together (create association). Strengthens both patterns.

Returns: 0 on success, -1 on error

Pattern Information

Get Pattern Info

int dagfs_get_pattern_info(pattern_id_t pattern_id, unsigned long *strength,
                          unsigned long *access_count, unsigned char *state);

Description: Get information about a pattern (strength, access count, state).

Returns: 0 on success, -1 on error

How It Works

Creating a "File" (Pattern)

Provide data and context tags
Allocate pattern metadata structure (64 bytes)
Allocate weight snapshot space in index storage
Copy data into weight snapshot (simplified storage)
Associate with tags for future lookup
Initialize pattern strength and access count

Reading a "File" (Pattern Reconstruction)

Query by tags or use pattern ID
Load associated weight configuration
Run forward pass through neural network to reconstruct full pattern
Strengthen weights (reconsolidation - read modifies!)
Increment access count
Return pointer to reconstructed data

Writing a "File" (Pattern Update)

Provide new data
Load existing weight configuration
Blend new data with existing weights (Hebbian learning)
Update weight snapshot
Update pattern strength (increase)
Increment access count

Deleting a "File" (Forgetting)

Load associated weight configuration
Weaken weights by ~10% (Long-Term Depression - LTD)
Remove tag associations
Mark pattern as deleted
Pattern fades away (weights approach zero over time)

Pattern Completion (Memory Recall)

Provide partial input (like a memory cue)
Load pattern's weight configuration
Pad partial input to full network input size
Run forward pass through neural network
Reconstruct full pattern from partial input
Strengthen pattern (reconsolidation)
Return completed pattern

Data Structures

Pattern Structure (64 bytes)

struct pattern {
    qword pattern_id;        // 8 bytes - Unique identifier
    qword tags_pointer;      // 8 bytes - Pointer to tag list
    qword nodes_pointer;      // 8 bytes - Pointer to node indices
    qword weights_pointer;    // 8 bytes - Pointer to weight snapshot
    qword strength;          // 8 bytes - Activation strength
    qword access_count;      // 8 bytes - How many times accessed
    byte  state;            // 1 byte - State (0=free, 1=active, 2=deleted)
    byte  file_type;        // 1 byte - File type (FILE_TYPE_EXECUTABLE, FILE_TYPE_DATA, etc.)
    word  file_flags;       // 2 bytes - File-specific flags
    byte  reserved[12];     // 12 bytes - Reserved for future use
};

File Types: EXECUTABLE (procedural memory), DATA (episodic/semantic memory), NEURAL, MEMORY, CONFIG, TEMP, SYMLINK, LIBRARY, DRIVER

See Also: File Philosophy for details on behavioral file types

Tag Structure (128 bytes)

struct tag {
    qword tag_id;           // 8 bytes - Unique identifier
    char  name[64];         // 64 bytes - Tag name string
    qword patterns_pointer;  // 8 bytes - Pointer to pattern list
    qword count;            // 8 bytes - Number of associated patterns
    qword strength;         // 8 bytes - Association strength
    byte  reserved[32];     // 32 bytes - Reserved for future use
};

Weight Snapshot

Each pattern stores a snapshot of its weight configuration:

Size: 1024 bytes (16 inputs × 8 hidden × 8 bytes per qword)
Location: Index storage (0x400000+)
Format: Array of 64-bit qwords

Memory Layout

Pattern Storage (0x300000 - 0x3FFFFF)

Size: 16MB

Contents:

Pattern metadata structures (64 bytes each)
Up to 1024 patterns supported
Pattern ID, tags, nodes, weights pointers
Pattern strength and access count

Index Storage (0x400000 - 0x4FFFFF)

Size: 16MB

Contents:

0x400000 - 0x40FFFF: Weight snapshots (1024 bytes per pattern) and text storage (256 bytes per pattern)
0x410000 - 0x41FFFF: Tag structures (128 bytes each, 512 tags × 128 bytes = 64KB)
0x420000 - 0x4FFFFF: Additional index storage for weight snapshots and text data

Tag Storage Details:

Base Address: 0x410000
Tag Structure Size: 128 bytes per tag
Max Tags: 512 tags (0x410000 - 0x41FFFF = 64KB)
Tag Structure: Tag ID (8 bytes), Name (64 bytes), Patterns pointer (8 bytes), Count (8 bytes), Strength (8 bytes), Reserved (32 bytes)

Ghost File Table (0x500000 - 0x50FFFF)

Size: 128KB

Status: ⏳ Planned for Phase 2

Contents:

Ghost file entries (128 bytes each)
Up to 1024 ghost file entries
Path → Pattern ID mapping
File type and flags

See Also: File Philosophy for details on ghost files and cues

Neural Network Weights (0x600000+)

Base Address: 0x600000 (6MB)

Contents:

Active weight matrices for 100k neuron network
Used during forward pass for pattern reconstruction
Sparse connectivity: ~1000 connections per neuron
Total size: ~784MB for all layer weights

Note: The 100k neuron network uses separate addresses for activations (0x5000000), biases (0x50C8000), and connection indices (0x5190000). See Memory Layout for complete network address map.

See Memory Layout for complete memory map.

Integration with Chapp-E

Neural Network

DAG-FS uses Chapp-E's existing neural network as its storage mechanism:

Nodes = Neurons in network layers (16 input, 8 hidden, 4 output)
Edges = Weights between layers (stored as 64-bit qwords)
Forward Pass = Pattern reconstruction mechanism
Weight Updates = Pattern storage/update via Hebbian learning
File: neural/networks/neural_network/network_64.asm

Brain Controller

Pattern operations are routed through the Brain Controller's message system:

System ID: SYS_ID_DAGFS (0x0D)
Message Types: Pattern operations (create, read, write, delete, query)
No Hierarchy: Flat message routing = associative access
Initialization: dagfs_init() called in braincontroller_init()

Neuromodulator System

Context-dependent access via neuromodulators:

Tags affect neuromodulator levels
Modulators influence pattern strength
Mood affects which patterns are accessible
Regional distribution affects pattern access

Working Memory

Short-term pattern buffer:

Active patterns during processing
Temporary associations
Pattern cache for recently accessed patterns

Example Usage

Traditional Filesystem

// Traditional approach
FILE *f = fopen("/home/user/memory.txt", "r");
fread(buffer, 1, size, f);
fclose(f);

DAG-FS: Creating a Pattern

// Create a pattern with tags
pattern_id_t pattern = dagfs_create_pattern(
    "first day of school",  // Tags
    memory_data,            // Data
    data_length             // Length
);

// Add additional tags
dagfs_add_tag(pattern, "chalk");
dagfs_add_tag(pattern, "happy");
dagfs_add_tag(pattern, "childhood");

DAG-FS: Reading a Pattern

// Query by tags
pattern_id_t pattern = dagfs_query_by_tag("first day of school");

// Reconstruct pattern (read - strengthens weights!)
void *data = dagfs_read_pattern(pattern);

// Pattern is now stronger due to reconsolidation

DAG-FS: Pattern Completion

// Complete pattern from partial input
char partial[] = "first day";
char output[1024];
int bytes = dagfs_complete_pattern(
    pattern,           // Pattern ID
    partial,           // Partial input
    strlen(partial),   // Input length
    output,            // Output buffer
    sizeof(output)     // Buffer size
);
// Output now contains full reconstructed pattern

DAG-FS: Updating a Pattern

// Update pattern (Hebbian learning)
dagfs_write_pattern(pattern, new_data, new_length);
// Weights are updated, pattern strength increases

DAG-FS: Associating Patterns

// Link two patterns together
dagfs_associate_patterns(pattern1, pattern2);
// Both patterns are strengthened

DAG-FS: Forgetting a Pattern

// Delete pattern (LTD - weakens weights)
dagfs_delete_pattern(pattern);
// Weights are weakened, pattern fades away

Implementation Details

Files

system/kernel/filesystem/dagfs.h - DAG-FS API and data structures
system/kernel/filesystem/dagfs.c - Core filesystem implementation (C-based)
system/kernel/neural/neuron.h - Neuron agent system (integrated with DAG-FS)
system/kernel/neural/neuron.c - Neuron agent implementation
system/kernel/core/neural_init.c - Neural system initialization

✅ MIGRATED TO C: DAG-FS has been completely rewritten in C for Chappie OS. The filesystem is now integrated directly into the kernel and initialized during kernel boot as part of the neural development sequence.

Build Integration

DAG-FS is integrated into the build system:

Assembled as separate object files
Linked into kernel binary
Initialized during Brain Controller startup

Performance Characteristics

Pattern Creation: O(1) - Direct allocation
Pattern Read: O(n) - Forward pass through network
Pattern Query: O(m) - Linear search through patterns
Tag Lookup: O(t) - Linear search through tags
Pattern Completion: O(n) - Forward pass with partial input

Benefits

Brain-Like: Matches actual neural memory storage mechanisms
Dynamic: Data changes with access (reconsolidation)
Associative: Access by context, not file paths
Distributed: No single point of failure
Efficient: Leverages existing neural network infrastructure
Scalable: Patterns can overlap and share nodes
Reconstructive: Patterns rebuilt from weights, not stored directly
Forgetting: Natural decay of unused patterns (LTD)

Implementation Status

Phase	Feature	Status
Phase 1	Basic structure and API	✅ Complete
Phase 2	Pattern storage in weights	✅ Complete
Phase 3	Pattern reconstruction	✅ Complete
Phase 4	Tag-based querying	✅ Complete
Phase 5	Hebbian learning integration	✅ Complete

🎉 PRODUCTION READY: All phases of DAG-FS are complete and fully functional. The system is integrated into Chapp-E's kernel and ready for use.

Neuroscience Background

DAG-FS is based on real neuroscience research:

Engrams: Memories stored as distributed patterns of synaptic strengths
Reconsolidation: Accessing a memory strengthens it (read modifies data)
Long-Term Potentiation (LTP): Strengthening of synapses (write operation)
Long-Term Depression (LTD): Weakening of synapses (delete operation)
Pattern Completion: Partial cues trigger full memory recall
Associative Memory: Memories linked by context, not location

DAG-FS: Dynamic Associative Graph Filesystem

Overview

Traditional Filesystem vs. DAG-FS

Core Concepts

1. Nodes (Neurons)

2. Edges (Synapses)

3. Patterns (Memories/Files)

4. Tags (Context/Associations)

Complete API Reference

Initialization

Pattern Operations

Create Pattern

Read Pattern

Write Pattern

Delete Pattern

Query Pattern

Tag Operations

Add Tag

Query by Tag

Pattern Completion

Complete Pattern

Associate Patterns

Pattern Information

Get Pattern Info

How It Works

Creating a "File" (Pattern)

Reading a "File" (Pattern Reconstruction)

Writing a "File" (Pattern Update)

Deleting a "File" (Forgetting)

Pattern Completion (Memory Recall)

Data Structures

Pattern Structure (64 bytes)

Tag Structure (128 bytes)

Weight Snapshot

Memory Layout

Pattern Storage (0x300000 - 0x3FFFFF)

Index Storage (0x400000 - 0x4FFFFF)

Ghost File Table (0x500000 - 0x50FFFF)

Neural Network Weights (0x600000+)

Integration with Chapp-E

Neural Network

Brain Controller

Neuromodulator System

Working Memory

Example Usage

Traditional Filesystem

DAG-FS: Creating a Pattern

DAG-FS: Reading a Pattern

DAG-FS: Pattern Completion

DAG-FS: Updating a Pattern

DAG-FS: Associating Patterns

DAG-FS: Forgetting a Pattern

Implementation Details

Files

Build Integration

Performance Characteristics

Benefits

Implementation Status

Neuroscience Background

Related Documentation