> ## Documentation Index
> Fetch the complete documentation index at: https://docs.heylua.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Data API

> Custom data storage with semantic vector search

## Overview

The Data API allows you to store custom data in collections with powerful semantic search capabilities using vector embeddings. Returns **DataEntryInstance** objects with direct property access.

```typescript theme={null}
import { Data } from 'lua-cli';

// Create with search indexing - returns DataEntryInstance
const entry = await Data.create('movies', {
  title: 'Inception',
  director: 'Christopher Nolan'
}, 'Inception Christopher Nolan sci-fi thriller dreams');

// Direct property access
console.log(entry.title);     // "Inception"
console.log(entry.director);  // "Christopher Nolan"
console.log(entry.id);        // "entry_abc123"

// Instance methods
await entry.update({ rating: 9.5 }, 'inception nolan 9.5 rating');
await entry.save();
await entry.delete();

// Semantic search - returns array of DataEntryInstance
const results = await Data.search('movies', 'mind-bending thriller', 10, 0.7);

// Direct array methods and property access
results.forEach(entry => {
  console.log(`${entry.title}: ${entry.score * 100}% match`);
});

const highScoring = results.filter(entry => entry.score > 0.8);
const titles = results.map(entry => entry.title);
```

<CardGroup cols={2}>
  <Card title="Direct Access" icon="bolt">
    Access `entry.title` not `entry.data.title`
  </Card>

  <Card title="Search Scores" icon="chart-line">
    Results include relevance scores
  </Card>

  <Card title="Instance Methods" icon="wrench">
    Built-in `update()`, `save()`, and `delete()`
  </Card>

  <Card title="Array Methods" icon="list">
    Use `.map()`, `.filter()` on search results
  </Card>
</CardGroup>

## Return Shape Reference

<Warning>
  **Each method returns a different type.** The most common mistake is treating `Data.search()` results like `Data.get()` results — they have different shapes.
</Warning>

| Method            | Returns                                             | How to use                                                                      |
| ----------------- | --------------------------------------------------- | ------------------------------------------------------------------------------- |
| `Data.create()`   | `DataEntryInstance` (Proxy)                         | `entry.fieldName` or `entry.data.fieldName`                                     |
| `Data.getEntry()` | `DataEntryInstance` (Proxy)                         | `entry.fieldName` or `entry.data.fieldName`                                     |
| `Data.search()`   | `DataEntryInstance[]` — **flat array, no envelope** | `results.map(e => e.fieldName)`, `results.length` — **no `.data`, no `.count`** |
| `Data.get()`      | `{ data: Entry[], pagination }` — **envelope**      | `result.data.map(e => e.data.fieldName)` — entries are raw, **not proxied**     |
| `Data.update()`   | `{ status, message }`                               | Check `result.status === 'success'`                                             |
| `Data.delete()`   | `{ status, message }`                               | Check `result.status === 'success'`                                             |

**Quick examples:**

```typescript theme={null}
// ✅ Data.search → flat array
const results = await Data.search('movies', 'thriller', 10, 0.7);
results.forEach(entry => console.log(entry.title, entry.score)); // Proxy works
console.log(results.length);          // ✅ count
// ❌ results.data   → undefined
// ❌ results.count  → undefined

// ✅ Data.get → { data, pagination } envelope
const page = await Data.get('movies', {}, 1, 20);
page.data.map(entry => entry.data.title); // raw entries: must use entry.data.field
console.log(page.pagination.totalPages);
// ❌ entry.title (without .data) → undefined on get() results

// ✅ Data.create / Data.getEntry → DataEntryInstance with Proxy
const entry = await Data.create('movies', { title: 'Inception' }, 'Inception Nolan');
console.log(entry.title);       // Proxy shortcut ✅
console.log(entry.data.title);  // Also works ✅
```

## Key Features

<CardGroup cols={2}>
  <Card title="Custom Collections" icon="database">
    Store any JSON data in named collections
  </Card>

  <Card title="Vector Search" icon="magnifying-glass">
    Semantic similarity search using AI embeddings
  </Card>

  <Card title="Flexible Schema" icon="diagram-project">
    No fixed schema - store any structure
  </Card>

  <Card title="Filtering" icon="filter">
    Query by field values with operators
  </Card>
</CardGroup>

## Methods

### create()

Create a new entry in a collection.

```typescript theme={null}
Data.create(
  collectionName: string,
  data: object,
  searchText?: string
): Promise<DataEntryInstance>
```

<ParamField path="collectionName" type="string" required>
  Name of the collection (e.g., 'movies', 'customers', 'articles')
</ParamField>

<ParamField path="data" type="object" required>
  Any JSON-serializable object to store
</ParamField>

<ParamField path="searchText" type="string">
  Text to index for vector search. Include all searchable content.
</ParamField>

**Returns:**

```typescript theme={null}
{
  id: string;
  data: object;
  createdAt: number;
  updatedAt: number;
  searchText?: string;
}
```

**Example:**

```typescript theme={null}
const movie = await Data.create('movies', {
  title: 'The Matrix',
  year: 1999,
  director: 'Wachowski Sisters',
  genre: 'Sci-Fi',
  rating: 8.7
}, 'The Matrix 1999 Wachowski sci-fi action cyberpunk reality virtual');

console.log(movie.id); // "entry_abc123"
```

### search()

Semantic search using vector embeddings.

```typescript theme={null}
Data.search(
  collectionName: string,
  searchText: string,
  limit?: number,
  scoreThreshold?: number
): Promise<DataEntryInstance[]>
```

<ParamField path="collectionName" type="string" required>
  Name of the collection to search
</ParamField>

<ParamField path="searchText" type="string" required>
  Search text (natural language query)
</ParamField>

<ParamField path="limit" type="number" default={10}>
  Maximum number of results to return
</ParamField>

<ParamField path="scoreThreshold" type="number" default={0.6}>
  Minimum similarity score (0-1). Higher = more similar.
</ParamField>

**Returns:** Array of `DataEntryInstance` objects, each with a `score` property.

**Similarity Scores:**

* `1.0` = Perfect match
* `0.8-0.9` = Very similar
* `0.6-0.7` = Somewhat similar
* `<0.6` = Low similarity

**Example:**

```typescript theme={null}
// Finds movies even if query doesn't match exact words!
const results = await Data.search('movies', 'mind-bending thriller', 5, 0.7);

results.forEach(entry => {
  console.log(`${entry.title} - Relevance: ${entry.score}`);
});
// Output:
// Inception - Relevance: 0.92
// The Matrix - Relevance: 0.85
// Interstellar - Relevance: 0.78
```

### get()

Retrieve entries with optional filtering and pagination.

```typescript theme={null}
Data.get(
  collectionName: string,
  filter?: object,
  page?: number,
  limit?: number
): Promise<GetCustomDataResponse>
```

<ParamField path="collectionName" type="string" required>
  Name of the collection
</ParamField>

<ParamField path="filter" type="object" default={{}}>
  Filter criteria (MongoDB-style queries)
</ParamField>

<ParamField path="page" type="number" default={1}>
  Page number (1-indexed)
</ParamField>

<ParamField path="limit" type="number" default={10}>
  Items per page
</ParamField>

**Returns:**

```typescript theme={null}
{
  data: Entry[];
  pagination: {
    currentPage: number;
    totalPages: number;
    totalCount: number;
    limit: number;
    hasNextPage: boolean;
    hasPrevPage: boolean;
    nextPage: number | null;
    prevPage: number | null;
  };
}
```

**Examples:**

```typescript theme={null}
// Get all
const all = await Data.get('movies');

// With pagination
const page2 = await Data.get('movies', {}, 2, 20);

// With filter
const recent = await Data.get('movies', {
  year: { $gte: 2020 }
});

// Complex filter
const sciFi = await Data.get('movies', {
  genre: 'Sci-Fi',
  rating: { $gte: 8.0 },
  year: { $gte: 2000, $lte: 2020 }
});
```

### getEntry()

Retrieve a specific entry by ID.

```typescript theme={null}
Data.getEntry(
  collectionName: string,
  entryId: string
): Promise<DataEntryInstance>
```

**Example:**

```typescript theme={null}
const movie = await Data.getEntry('movies', 'entry_abc123');
console.log(movie.title); // "Inception" (direct property access via proxy)
```

### update()

Update an existing entry.

```typescript theme={null}
Data.update(
  collectionName: string,
  entryId: string,
  data: object,
  searchText?: string
): Promise<UpdateCustomDataResponse>
```

<ParamField path="collectionName" type="string" required>
  Name of the collection
</ParamField>

<ParamField path="entryId" type="string" required>
  ID of the entry to update
</ParamField>

<ParamField path="data" type="object" required>
  Data to merge with existing entry
</ParamField>

<ParamField path="searchText" type="string">
  Optional new text for vector search indexing
</ParamField>

**Returns:**

```typescript theme={null}
{
  status: string;   // "success"
  message: string;  // "Custom data entry updated"
}
```

**Example:**

```typescript theme={null}
// Update data and search text
const result = await Data.update('movies', 'entry_abc123', {
  rating: 8.8,      // Updated rating
  awards: ['Oscar'] // New field
}, 'Inception Christopher Nolan oscar winner');

// Update data only
await Data.update('movies', 'entry_abc123', {
  views: 1500
});
```

<Note>
  Updates merge with existing data. Only the specified fields are updated; other fields are preserved.
</Note>

## DataEntryInstance Methods

When you retrieve or create data entries, you get a `DataEntryInstance` object with convenient instance methods.

### save()

Save the current state of the data entry to the server. This is a convenience method that persists all changes made to the entry.

```typescript theme={null}
entry.save(searchText?: string): Promise<boolean>
```

<ParamField path="searchText" type="string">
  Optional new text for vector search indexing
</ParamField>

**Returns:** Promise resolving to `true` if successful

**Example:**

```typescript theme={null}
const entry = await Data.getEntry('movies', 'entry_abc123');

// Modify properties directly
entry.title = "Inception";
entry.rating = 9.0;
entry.year = 2010;

// Save all changes at once
await entry.save();

// Or save with updated search text
await entry.save('Inception 2010 Nolan sci-fi thriller dreams');
```

<Note>
  The `save()` method provides a simpler workflow - modify properties then save, rather than calling `Data.update()` with the collection name and entry ID.
</Note>

### update() (Instance Method)

Update the entry using the instance method.

```typescript theme={null}
entry.update(data: object, searchText?: string): Promise<object>
```

**Returns:** Promise resolving to the updated data object

**Example:**

```typescript theme={null}
const entry = await Data.getEntry('movies', 'entry_abc123');

// Update with new search text
const updatedData = await entry.update(
  { rating: 9.5, review: 'Mind-bending masterpiece' }, 
  'inception nolan 9.5 rating masterpiece'
);

console.log(updatedData.rating); // 9.5

// Update data only (keeps existing search text)
await entry.update({ views: 1000 });
```

### delete() (Instance Method)

Delete the entry using the instance method.

```typescript theme={null}
entry.delete(): Promise<boolean>
```

**Example:**

```typescript theme={null}
const entry = await Data.getEntry('movies', 'entry_abc123');
await entry.delete();
```

## Static Methods

### delete()

Delete an entry using the static method.

```typescript theme={null}
Data.delete(
  collectionName: string,
  entryId: string
): Promise<DeleteCustomDataResponse>
```

**Example:**

```typescript theme={null}
await Data.delete('movies', 'entry_abc123');
```

## Use Cases

### Knowledge Base

```typescript theme={null}
export class CreateArticleTool implements LuaTool {
  async execute(input: any) {
    // Create searchable article
    const article = await Data.create('kb_articles', {
      title: input.title,
      content: input.content,
      category: input.category,
      tags: input.tags
    }, `${input.title} ${input.content} ${input.tags.join(' ')}`);
    
    return { articleId: article.id };
  }
}

export class SearchArticlesTool implements LuaTool {
  async execute(input: any) {
    // Semantic search
    const results = await Data.search(
      'kb_articles',
      input.query,
      10,
      0.7
    );
    
    return {
      articles: results.map(entry => ({
        id: entry.id,
        title: entry.title,
        content: entry.content.substring(0, 200),
        relevance: Math.round(entry.score * 100) + '%'
      }))
    };
  }
}
```

### Customer CRM

```typescript theme={null}
export class CreateCustomerTool implements LuaTool {
  async execute(input: any) {
    const customer = await Data.create('customers', {
      name: input.name,
      email: input.email,
      company: input.company,
      status: 'active',
      createdAt: new Date().toISOString()
    }, `${input.name} ${input.company} ${input.email}`);
    
    // Log interaction
    await Data.create('interactions', {
      customerId: customer.id,
      type: 'created',
      notes: 'Initial contact',
      timestamp: new Date().toISOString()
    });
    
    return { customerId: customer.id };
  }
}
```

### Task Management

```typescript theme={null}
export class CreateTaskTool implements LuaTool {
  async execute(input: any) {
    const task = await Data.create('tasks', {
      title: input.title,
      description: input.description,
      status: 'pending',
      priority: input.priority,
      createdAt: new Date().toISOString()
    }, `${input.title} ${input.description}`);
    
    return { taskId: task.id };
  }
}

export class SearchTasksTool implements LuaTool {
  async execute(input: any) {
    const results = await Data.search('tasks', input.query, 20, 0.6);
    
    return {
      tasks: results.map(entry => ({
        id: entry.id,
        title: entry.title,
        description: entry.description,
        status: entry.status,
        priority: entry.priority,
        relevance: entry.score
      }))
    };
  }
}
```

## Filter Operators

The Data API supports MongoDB-style filter operators:

```typescript theme={null}
// Comparison
{ age: { $eq: 25 } }       // Equal
{ age: { $ne: 25 } }       // Not equal
{ age: { $gt: 25 } }       // Greater than
{ age: { $gte: 25 } }      // Greater than or equal
{ age: { $lt: 25 } }       // Less than
{ age: { $lte: 25 } }      // Less than or equal

// Logical
{ $and: [{ age: { $gte: 18 } }, { age: { $lte: 65 } }] }
{ $or: [{ status: 'active' }, { status: 'pending' }] }

// Array
{ tags: { $in: ['urgent', 'important'] } }
{ tags: { $nin: ['spam', 'archived'] } }

// Existence
{ email: { $exists: true } }
```

## Best Practices

<AccordionGroup>
  <Accordion title="Create Rich Search Text">
    Include all searchable content in searchText:

    ```typescript theme={null}
    const searchText = [
      item.title,
      item.description,
      item.category,
      item.tags.join(' '),
      item.author
    ].filter(Boolean).join(' ');

    await Data.create('items', item, searchText);
    ```
  </Accordion>

  <Accordion title="Use Appropriate Score Thresholds">
    * `0.8+`: High precision, few results
    * `0.7`: Balanced (recommended default)
    * `0.6`: More results, lower precision
    * `<0.6`: May return irrelevant results
  </Accordion>

  <Accordion title="Structure Data Consistently">
    Use consistent field names across entries:

    ```typescript theme={null}
    // ✅ Good - Consistent
    await Data.create('items', {
      name: 'Item 1',
      price: 10.99,
      inStock: true
    });

    // ❌ Bad - Inconsistent
    await Data.create('items', {
      title: 'Item 2',  // Different field name
      cost: 15.99,      // Different field name
      available: true   // Different field name
    });
    ```
  </Accordion>

  <Accordion title="Add Timestamps">
    Track when entries are created/modified:

    ```typescript theme={null}
    await Data.create('items', {
      ...data,
      createdAt: new Date().toISOString(),
      updatedAt: new Date().toISOString()
    });
    ```
  </Accordion>
</AccordionGroup>

## Vector Search Tips

<Tabs>
  <Tab title="What It Is">
    Vector search uses AI to understand meaning, not just match keywords.

    **Example:**

    * Query: "affordable laptop for students"
    * Finds: "budget-friendly notebook for college"
    * Even though no words match exactly!
  </Tab>

  <Tab title="Best For">
    * Knowledge bases
    * FAQs
    * Product recommendations
    * Content discovery
    * Document search
  </Tab>

  <Tab title="Tips">
    1. Include synonyms in searchText
    2. Use natural language queries
    3. Adjust threshold based on results
    4. Test with real user queries
  </Tab>
</Tabs>

<Note title="Debugging tip">
  If your Data results don't look right, log the raw return value before transforming it:

  ```typescript theme={null}
  const results = await Data.search('collection', input.query);
  console.log('Data.search result:', JSON.stringify(results, null, 2));
  ```

  Then run `lua logs --type skill --limit 5` after sending a test message to see the actual shape at runtime. See the [Debugging Skills guide](/cli/debugging) for the full 5-step workflow.
</Note>

## Next Steps

<CardGroup cols={2}>
  <Card title="Custom Data Examples" icon="layer-group" href="/examples/custom-data">
    See working examples
  </Card>

  <Card title="Build Your First Skill" icon="hammer" href="/getting-started/first-skill">
    Complete tutorial using Data API
  </Card>

  <Card title="Debugging Skills" icon="bug" href="/cli/debugging">
    Inspect runtime return values
  </Card>
</CardGroup>