> ## Documentation Index
> Fetch the complete documentation index at: https://docs.heylua.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AI API

> Generate AI responses with custom context in your tools

## Overview

The AI API provides isolated text generation from within tools, aligned with [Vercel AI SDK `generateText`](https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-text) semantics. Requests are proxied to a dedicated generation endpoint — they do **not** go through the agent chat pipeline.

```typescript theme={null}
import { AI } from 'lua-cli';

// Quick text generation
const text = await AI.generate('Summarize the latest AI news.');

// Full options with rich result
const result = await AI.generate({
  model: 'google/gemini-2.0-flash',
  system: 'You are concise.',
  prompt: 'What is the weather in London?',
});
```

<Note>
  **Powerful Tool Enhancement:** The AI API lets you embed AI generation directly in your tools with custom personas, multi-modal inputs, and rich response metadata.
</Note>

## Import

```typescript theme={null}
import { AI } from 'lua-cli';
// or
import { AI } from 'lua-cli/skill';
```

## Capabilities

<CardGroup cols={2}>
  <Card title="Text Generation" icon="file-lines">
    Generate content with custom prompts and personas
  </Card>

  <Card title="Image Analysis" icon="image">
    Analyze images with AI vision capabilities
  </Card>

  <Card title="Document Processing" icon="file-pdf">
    Process and analyze documents with AI
  </Card>

  <Card title="Multi-Modal" icon="layer-group">
    Combine text, images, and files in one request
  </Card>

  <Card title="Multi-Provider" icon="server">
    Google (Vertex AI), OpenAI, and Anthropic models
  </Card>

  <Card title="Web Grounding" icon="globe">
    Google Search grounding with source URLs
  </Card>
</CardGroup>

## Methods

### Simplified: AI.generate(prompt, content?)

Quick text generation. Returns plain text as a string.

<ParamField path="prompt" type="string" required>
  When called with one argument, this is the user prompt. When called with two arguments, it becomes the system instruction.
</ParamField>

<ParamField path="content" type="UserContent">
  User message content. Accepts a string or an array of multimodal parts (`TextPart`, `ImagePart`, `FilePart`) from the AI SDK.
</ParamField>

**Returns:** `Promise<string>`

```typescript theme={null}
// Single argument: prompt is the user message
const text = await AI.generate('Summarize the latest AI news.');

// Two arguments: system instruction + user content
const text2 = await AI.generate(
  'You are a helpful assistant.',
  [{ type: 'text', text: 'What products do you recommend?' }]
);

// Multi-modal content
const analysis = await AI.generate(
  'You are an image analysis expert.',
  [
    { type: 'text', text: 'What do you see?' },
    { type: 'image', url: 'https://example.com/photo.jpg' }
  ]
);
```

### Full options: AI.generate(options)

Full control over generation parameters. Returns a rich result object.

<ParamField path="model" type="string">
  Model to use, e.g. `'google/gemini-2.0-flash'`, `'openai/gpt-4o'`, `'anthropic/claude-sonnet-4-20250514'`. Defaults to the agent's configured model.
</ParamField>

<ParamField path="system" type="string">
  System instruction.
</ParamField>

<ParamField path="prompt" type="string">
  User prompt (simple text).
</ParamField>

<ParamField path="messages" type="array">
  Conversation messages (AI SDK `ModelMessage[]`).
</ParamField>

<ParamField path="temperature" type="number">
  Sampling temperature (0–2).
</ParamField>

<ParamField path="maxOutputTokens" type="number">
  Maximum tokens to generate.
</ParamField>

<ParamField path="structuredOutput" type="{ schema: JSONSchema7 }">
  Constrain the model response to a JSON Schema. The parsed object lands on
  `result.output`. Mirrors AI SDK `Output.object({ schema })`. Object-mode
  only — array / choice modes are future extensions.

  When set on a Google model, the auto-injected `google_search` tool is
  suppressed (Vertex does not allow mixing function-calling tools with
  `google_search` in one request).
</ParamField>

**Returns:** `Promise<AiGenerateOutput>`

```typescript theme={null}
const result = await AI.generate({
  model: 'google/gemini-2.0-flash',
  system: 'You are concise.',
  prompt: 'What is the weather in London?',
  temperature: 0.7,
});

result.text           // Generated text
result.finishReason   // 'stop', 'length', etc.
result.usage          // { promptTokens, completionTokens, totalTokens }
result.sources        // Google Search grounding URLs
```

### Response Shape (AiGenerateOutput)

The full-options response mirrors AI SDK `GenerateTextResult`:

| Field            | Type                     | Description                                                                                                          |
| ---------------- | ------------------------ | -------------------------------------------------------------------------------------------------------------------- |
| `text`           | `string`                 | Generated text                                                                                                       |
| `finishReason`   | `FinishReason`           | `'stop'`, `'length'`, `'content-filter'`, `'tool-calls'`, `'error'`, `'other'`, `'unknown'`                          |
| `usage`          | `LanguageModelUsage`     | `{ promptTokens, completionTokens, totalTokens }`                                                                    |
| `reasoning?`     | `ReasoningOutput[]`      | Model reasoning steps (e.g. Gemini thinking)                                                                         |
| `reasoningText?` | `string`                 | Concatenated reasoning text                                                                                          |
| `sources?`       | `AiGenerateSource[]`     | URL sources from Google Search grounding                                                                             |
| `toolCalls?`     | `AiGenerateToolCall[]`   | Tool calls made during generation                                                                                    |
| `toolResults?`   | `AiGenerateToolResult[]` | Tool results from generation                                                                                         |
| `output?`        | `unknown`                | Parsed structured result when `structuredOutput` was set on the request. Shape conforms to the supplied JSON Schema. |
| `warnings?`      | `CallWarning[]`          | Provider warnings                                                                                                    |

## Supported Providers

| Provider           | Prefix       | Example                              |
| ------------------ | ------------ | ------------------------------------ |
| Google (Vertex AI) | `google/`    | `google/gemini-2.0-flash`            |
| OpenAI             | `openai/`    | `openai/gpt-4o`                      |
| Anthropic          | `anthropic/` | `anthropic/claude-sonnet-4-20250514` |

If the requested provider's API key is not configured, the request falls back to the default Vertex AI model.

<Note>
  Google models automatically get **Google Search grounding** — real-time web search results appear in the `sources` field of the full-options response.
</Note>

## Content Types

### Text

```typescript theme={null}
[{ type: 'text', text: 'Your message here' }]
```

### Image

```typescript theme={null}
[
  { type: 'text', text: 'What do you see in this image?' },
  { type: 'image', url: 'https://example.com/photo.jpg' }
]
```

### File

```typescript theme={null}
[
  { type: 'text', text: 'Summarize this document' },
  { type: 'file', url: 'https://example.com/doc.pdf', mimeType: 'application/pdf' }
]
```

## Complete Examples

### Product Description Generator

```typescript theme={null}
import { LuaTool, AI, Products } from 'lua-cli/skill';
import { z } from 'zod';

export default class GenerateDescriptionTool implements LuaTool {
  name = 'generate_product_description';
  description = 'Generate compelling product descriptions using AI';
  
  inputSchema = z.object({
    productId: z.string(),
    style: z.enum(['casual', 'professional', 'luxury']).optional()
  });
  
  async execute(input: z.infer<typeof this.inputSchema>) {
    const product = await Products.getById(input.productId);
    if (!product) {
      return { success: false, error: 'Product not found' };
    }
    
    const style = input.style || 'professional';
    const description = await AI.generate(
      `You are a ${style} copywriter. Create a 2-3 sentence product description.`,
      [{ type: 'text', text: `Product: ${product.name}, Price: $${product.price}` }]
    );
    
    await product.update({ description });
    return { success: true, productId: input.productId, description };
  }
}
```

### Weather Search with Google Grounding

```typescript theme={null}
import { LuaTool, AI } from 'lua-cli/skill';
import { z } from 'zod';

export default class WeatherSearchTool implements LuaTool {
  name = 'search_weather';
  description = 'Search current weather using AI with Google Search grounding';
  
  inputSchema = z.object({
    location: z.string().describe('City or location')
  });
  
  async execute(input: z.infer<typeof this.inputSchema>) {
    const result = await AI.generate({
      model: 'google/gemini-2.0-flash',
      system: 'You report current weather conditions concisely.',
      prompt: `What is the current weather in ${input.location}?`,
    });

    return {
      weather: result.text,
      sources: result.sources?.map(s => ({ title: s.title, url: s.url })) ?? [],
      usage: result.usage,
    };
  }
}
```

### Image Analysis Tool

```typescript theme={null}
import { LuaTool, AI } from 'lua-cli/skill';
import { z } from 'zod';

export default class AnalyzeImageTool implements LuaTool {
  name = 'analyze_image';
  description = 'Analyze images using AI vision';
  
  inputSchema = z.object({
    imageUrl: z.string().url(),
    question: z.string().optional()
  });
  
  async execute(input: z.infer<typeof this.inputSchema>) {
    const analysis = await AI.generate(
      'You are an image analysis expert. Describe what you see in detail.',
      [
        { type: 'text', text: input.question || 'Describe this image in detail' },
        { type: 'image', url: input.imageUrl }
      ]
    );
    
    return { success: true, imageUrl: input.imageUrl, analysis };
  }
}
```

### Content Summarizer

```typescript theme={null}
import { LuaTool, AI } from 'lua-cli/skill';
import { z } from 'zod';

export default class SummarizeTool implements LuaTool {
  name = 'summarize_content';
  description = 'Summarize long text content';
  
  inputSchema = z.object({
    content: z.string(),
    maxLength: z.enum(['brief', 'medium', 'detailed']).optional()
  });
  
  async execute(input: z.infer<typeof this.inputSchema>) {
    const instructions = {
      brief: 'in 1-2 sentences',
      medium: 'in 1 paragraph',
      detailed: 'in 2-3 paragraphs with key points'
    };
    const length = input.maxLength || 'medium';
    
    const summary = await AI.generate(
      `You are a professional content summarizer. Create a clear summary ${instructions[length]}.`,
      [{ type: 'text', text: `Summarize:\n\n${input.content}` }]
    );
    
    return { success: true, summary, originalLength: input.content.length };
  }
}
```

### Translation Tool

```typescript theme={null}
import { LuaTool, AI } from 'lua-cli/skill';
import { z } from 'zod';

export default class TranslateTool implements LuaTool {
  name = 'translate_text';
  description = 'Translate text to different languages';
  
  inputSchema = z.object({
    text: z.string(),
    targetLanguage: z.string(),
    sourceLanguage: z.string().optional()
  });
  
  async execute(input: z.infer<typeof this.inputSchema>) {
    const translation = await AI.generate(
      `You are a professional translator. Translate accurately to ${input.targetLanguage}. Return ONLY the translated text.`,
      [{ type: 'text', text: input.text }]
    );
    
    return {
      success: true,
      original: input.text,
      translation,
      from: input.sourceLanguage || 'auto-detect',
      to: input.targetLanguage
    };
  }
}
```

### Sentiment Analysis (Structured Output)

Use `structuredOutput` to make the model return an object that conforms to a
JSON Schema. The parsed result lands on `result.output` — no more
`JSON.parse(...)` and no more "the model wrapped it in markdown again" bugs.

```typescript theme={null}
import { LuaTool, AI } from 'lua-cli/skill';
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

const SentimentSchema = z.object({
  sentiment: z.enum(['positive', 'negative', 'neutral']),
  score: z.number().min(0).max(1),
  summary: z.string(),
});

export default class SentimentAnalysisTool implements LuaTool {
  name = 'analyze_sentiment';
  description = 'Analyze the sentiment of text';

  inputSchema = z.object({ text: z.string() });

  async execute(input: z.infer<typeof this.inputSchema>) {
    const result = await AI.generate({
      system: 'You are a sentiment analysis expert.',
      prompt: `Analyze:\n\n"${input.text}"`,
      temperature: 0,
      structuredOutput: {
        schema: zodToJsonSchema(SentimentSchema) as Record<string, unknown>,
      },
    });

    const parsed = SentimentSchema.safeParse(result.output);
    if (!parsed.success) {
      return { success: false, error: 'Schema mismatch', rawResponse: result.text };
    }

    return { success: true, ...parsed.data, originalText: input.text };
  }
}
```

## Best Practices

<AccordionGroup>
  <Accordion title="Provide clear, specific context">
    Good context leads to better results

    ```typescript theme={null}
    // Good — specific and clear
    const context = `You are a product reviewer.
    Rate products on Quality (1-10), Value (1-10), Features (1-10).
    Return ONLY a JSON object with these ratings.`;

    // Bad — vague
    const context = `Review this product`;
    ```
  </Accordion>

  <Accordion title="Choose the right overload">
    Use simplified for quick text, full options when you need metadata

    ```typescript theme={null}
    // Simplified — just need the text
    const summary = await AI.generate('Summarize this article.', content);

    // Full options — need usage stats, sources, finish reason
    const result = await AI.generate({
      model: 'google/gemini-2.0-flash',
      prompt: 'What is happening in tech today?',
    });
    console.log(result.sources); // Google Search grounding URLs
    ```
  </Accordion>

  <Accordion title="Handle variable responses">
    AI responses may vary — always validate

    ```typescript theme={null}
    try {
      const response = await AI.generate(context, messages);
      if (!response || response.trim().length === 0) {
        return { success: false, error: 'Empty response' };
      }
      return { success: true, response };
    } catch (error) {
      return { success: false, error: error.message };
    }
    ```
  </Accordion>

  <Accordion title="Don't use AI for simple logic">
    Use regular code for deterministic tasks

    ```typescript theme={null}
    // Bad — waste of AI for simple math
    await AI.generate('Calculate 2 + 2');

    // Good — use regular code
    const result = 2 + 2;
    ```
  </Accordion>
</AccordionGroup>

## Common Use Cases

| Use Case               | Example Tool                                           |
| ---------------------- | ------------------------------------------------------ |
| **Content generation** | Product descriptions, email drafts, social media posts |
| **Analysis**           | Sentiment analysis, image recognition, document review |
| **Translation**        | Multi-language support                                 |
| **Summarization**      | Long documents, articles, conversations                |
| **Recommendations**    | Product suggestions, content curation                  |
| **Moderation**         | Content filtering, safety checks                       |
| **Q\&A**               | Document queries, knowledge retrieval                  |
| **Web search**         | Real-time information via Google Search grounding      |

## Performance Considerations

<CardGroup cols={2}>
  <Card title="Response Time" icon="clock">
    AI generation typically takes 1-5 seconds. Longer/more complex requests take more time.
  </Card>

  <Card title="Context Length" icon="file">
    Maximum context length varies by model. Keep contexts focused and relevant.
  </Card>

  <Card title="Image Processing" icon="image">
    Image analysis is slower than text. Optimize image sizes when possible.
  </Card>

  <Card title="Caching" icon="database">
    Cache common AI responses. Store frequently requested generations.
  </Card>
</CardGroup>

## Limitations

* Maximum context length varies by model
* Image size limits apply (optimize before sending)
* File types supported: PDF, images, text files
* Response may vary between calls (non-deterministic)
* Some content may be filtered for safety
* Processing time increases with input complexity

## Error Handling

```typescript theme={null}
async execute(input: any) {
  try {
    const response = await AI.generate(context, messages);
    
    if (!response || response.trim().length === 0) {
      return { success: false, error: 'AI returned empty response' };
    }
    
    return { success: true, response };
  } catch (error) {
    return {
      success: false,
      error: error instanceof Error ? error.message : 'AI generation failed'
    };
  }
}
```

## Related APIs

<CardGroup cols={2}>
  <Card title="User API" href="/api/user" icon="user">
    Get user context for personalization
  </Card>

  <Card title="Data API" href="/api/data" icon="database">
    Store AI-generated content
  </Card>

  <Card title="Products API" href="/api/products" icon="box">
    Enhance products with AI descriptions
  </Card>

  <Card title="Jobs API" href="/api/jobs" icon="clock">
    Schedule AI generation tasks
  </Card>
</CardGroup>

## See Also

* [LuaTool](/api/luatool) - Creating tools
* [Tool Examples](/examples/overview) - More tool patterns
* [User API](/api/user) - Accessing user data