> ## Documentation Index
> Fetch the complete documentation index at: https://docs.heylua.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Mac Controller

> Control your MacBook remotely through your AI agent

## Overview

**Use case:** Personal computer automation via WhatsApp or web chat

Turn your Mac into an AI-controllable device. Ask your agent to take screenshots, open apps, search files, control music, manage your clipboard, and more -- all from a chat message.

**Commands:**

* `take_screenshot` -- Capture the screen and return the image
* `send_notification` -- Show a desktop notification
* `open_url` -- Open a URL in the default browser
* `open_app` -- Launch an application by name
* `search_files` -- Find files using Spotlight
* `get_active_app` -- Get the currently focused application
* `system_info` -- Hostname, CPU, memory, uptime, battery
* `get_clipboard` -- Read clipboard contents
* `set_clipboard` -- Write text to the clipboard
* `lock_screen` -- Lock the Mac
* `play_music` -- Play, pause, or skip tracks in Apple Music or Spotify
* `set_volume` -- Set the system volume (0--100)
* `run_shortcut` -- Run a Shortcuts.app shortcut by name

<Note>
  **Prerequisites:**

  * [Node.js](https://nodejs.org/) 18+ (for the Node.js client) or Python 3.10+ (for the Python client)
  * `terminal-notifier` for desktop notifications -- install with `brew install terminal-notifier`
  * Screen recording permission granted to your terminal app (for screenshots)
</Note>

## Device Client

<CodeGroup>
  ```typescript Node.js theme={null}
  import { DeviceClient } from '@lua-ai-global/device-client';
  import { execSync } from 'child_process';
  import { readFileSync, unlinkSync } from 'fs';
  import { tmpdir } from 'os';
  import { join } from 'path';

  const device = new DeviceClient({
    agentId: process.env.LUA_AGENT_ID!,
    apiKey: process.env.LUA_API_KEY!,
    deviceName: 'my-macbook',
    group: 'personal-devices',
    commands: [
      {
        name: 'take_screenshot',
        description: 'Capture a screenshot of the entire screen and return the image URL.',
        inputSchema: { type: 'object', properties: {} },
      },
      {
        name: 'send_notification',
        description: 'Show a macOS desktop notification.',
        inputSchema: {
          type: 'object',
          properties: {
            title: { type: 'string', description: 'Notification title' },
            message: { type: 'string', description: 'Notification body text' },
          },
          required: ['title', 'message'],
        },
      },
      {
        name: 'open_url',
        description: 'Open a URL in the default browser.',
        inputSchema: {
          type: 'object',
          properties: {
            url: { type: 'string', description: 'The URL to open' },
          },
          required: ['url'],
        },
      },
      {
        name: 'open_app',
        description: 'Launch an application by name (e.g., "Slack", "Safari", "Terminal").',
        inputSchema: {
          type: 'object',
          properties: {
            name: { type: 'string', description: 'Application name' },
          },
          required: ['name'],
        },
      },
      {
        name: 'search_files',
        description: 'Search for files by name using Spotlight (mdfind).',
        inputSchema: {
          type: 'object',
          properties: {
            query: { type: 'string', description: 'Filename or search term' },
            limit: { type: 'number', description: 'Max results (default 10)' },
          },
          required: ['query'],
        },
      },
      {
        name: 'get_active_app',
        description: 'Get the name of the currently focused application.',
        inputSchema: { type: 'object', properties: {} },
      },
      {
        name: 'system_info',
        description: 'Get system information: hostname, CPU, memory, uptime, and battery level.',
        inputSchema: { type: 'object', properties: {} },
      },
      {
        name: 'get_clipboard',
        description: 'Read the current clipboard text contents.',
        inputSchema: { type: 'object', properties: {} },
      },
      {
        name: 'set_clipboard',
        description: 'Set the clipboard text contents.',
        inputSchema: {
          type: 'object',
          properties: {
            text: { type: 'string', description: 'Text to copy to clipboard' },
          },
          required: ['text'],
        },
      },
      {
        name: 'lock_screen',
        description: 'Lock the Mac screen immediately.',
        inputSchema: { type: 'object', properties: {} },
      },
      {
        name: 'play_music',
        description: 'Control music playback: play, pause, or skip to next/previous track.',
        inputSchema: {
          type: 'object',
          properties: {
            action: {
              type: 'string',
              enum: ['play', 'pause', 'next', 'previous'],
              description: 'Playback action',
            },
            app: {
              type: 'string',
              enum: ['Music', 'Spotify'],
              description: 'Music app to control (default: Music)',
            },
          },
          required: ['action'],
        },
      },
      {
        name: 'set_volume',
        description: 'Set the system output volume.',
        inputSchema: {
          type: 'object',
          properties: {
            level: { type: 'number', minimum: 0, maximum: 100, description: 'Volume level 0-100' },
          },
          required: ['level'],
        },
      },
      {
        name: 'run_shortcut',
        description: 'Run a macOS Shortcuts.app shortcut by name.',
        inputSchema: {
          type: 'object',
          properties: {
            name: { type: 'string', description: 'Shortcut name' },
          },
          required: ['name'],
        },
      },
    ],
  });

  device.onCommand('take_screenshot', async () => {
    const filepath = join(tmpdir(), `screenshot-${Date.now()}.png`);
    execSync(`screencapture -x ${filepath}`);
    const buffer = readFileSync(filepath);
    const url = await device.uploadFile(buffer, `screenshot-${Date.now()}.png`, 'image/png');
    unlinkSync(filepath);
    return { imageUrl: url, timestamp: new Date().toISOString() };
  });

  device.onCommand('send_notification', async (payload) => {
    const title = payload.title.replace(/"/g, '\\"');
    const message = payload.message.replace(/"/g, '\\"');
    execSync(`terminal-notifier -title "${title}" -message "${message}"`);
    return { sent: true, title: payload.title, message: payload.message };
  });

  device.onCommand('open_url', async (payload) => {
    execSync(`open "${payload.url}"`);
    return { opened: true, url: payload.url };
  });

  device.onCommand('open_app', async (payload) => {
    execSync(`open -a "${payload.name}"`);
    return { opened: true, app: payload.name };
  });

  device.onCommand('search_files', async (payload) => {
    const limit = payload.limit || 10;
    const raw = execSync(`mdfind -name "${payload.query}" | head -${limit}`).toString().trim();
    const files = raw ? raw.split('\n') : [];
    return { query: payload.query, results: files, count: files.length };
  });

  device.onCommand('get_active_app', async () => {
    const script = 'tell application "System Events" to get name of first application process whose frontmost is true';
    const name = execSync(`osascript -e '${script}'`).toString().trim();
    return { activeApp: name };
  });

  device.onCommand('system_info', async () => {
    const hostname = execSync('hostname').toString().trim();
    const cpu = execSync('sysctl -n machdep.cpu.brand_string').toString().trim();
    const memBytes = parseInt(execSync('sysctl -n hw.memsize').toString().trim());
    const memGB = Math.round(memBytes / 1073741824);
    const uptime = execSync('uptime').toString().trim();
    let battery = 'N/A';
    try {
      battery = execSync('pmset -g batt | grep -o "[0-9]*%"').toString().trim();
    } catch {}
    return { hostname, cpu, memoryGB: memGB, uptime, battery };
  });

  device.onCommand('get_clipboard', async () => {
    const text = execSync('osascript -e "the clipboard"').toString().trim();
    return { clipboard: text };
  });

  device.onCommand('set_clipboard', async (payload) => {
    const escaped = payload.text.replace(/"/g, '\\"');
    execSync(`osascript -e 'set the clipboard to "${escaped}"'`);
    return { set: true, text: payload.text };
  });

  device.onCommand('lock_screen', async () => {
    execSync('pmset displaysleepnow');
    return { locked: true, timestamp: new Date().toISOString() };
  });

  device.onCommand('play_music', async (payload) => {
    const app = payload.app || 'Music';
    const actionMap: Record<string, string> = {
      play: 'play', pause: 'pause', next: 'next track', previous: 'previous track',
    };
    const command = actionMap[payload.action];
    execSync(`osascript -e 'tell application "${app}" to ${command}'`);
    return { action: payload.action, app };
  });

  device.onCommand('set_volume', async (payload) => {
    const vol = Math.round((payload.level / 100) * 7);
    execSync(`osascript -e 'set volume output volume ${payload.level}'`);
    return { volume: payload.level };
  });

  device.onCommand('run_shortcut', async (payload) => {
    execSync(`shortcuts run "${payload.name}"`);
    return { ran: true, shortcut: payload.name };
  });

  async function main() {
    await device.connect();
    console.log('Mac controller online');
  }

  main().catch(console.error);
  ```

  ```python Python theme={null}
  import asyncio
  import os
  import subprocess
  import tempfile
  from datetime import datetime, timezone
  from lua_device import DeviceClient, DeviceCommandDefinition

  client = DeviceClient(
      agent_id=os.environ["LUA_AGENT_ID"],
      api_key=os.environ["LUA_API_KEY"],
      device_name="my-macbook",
      group="personal-devices",
      commands=[
          DeviceCommandDefinition(
              name="take_screenshot",
              description="Capture a screenshot of the entire screen and return the image URL.",
              input_schema={"type": "object", "properties": {}},
          ),
          DeviceCommandDefinition(
              name="send_notification",
              description="Show a macOS desktop notification.",
              input_schema={
                  "type": "object",
                  "properties": {
                      "title": {"type": "string", "description": "Notification title"},
                      "message": {"type": "string", "description": "Notification body text"},
                  },
                  "required": ["title", "message"],
              },
          ),
          DeviceCommandDefinition(
              name="open_url",
              description="Open a URL in the default browser.",
              input_schema={
                  "type": "object",
                  "properties": {
                      "url": {"type": "string", "description": "The URL to open"},
                  },
                  "required": ["url"],
              },
          ),
          DeviceCommandDefinition(
              name="open_app",
              description="Launch an application by name (e.g., 'Slack', 'Safari', 'Terminal').",
              input_schema={
                  "type": "object",
                  "properties": {
                      "name": {"type": "string", "description": "Application name"},
                  },
                  "required": ["name"],
              },
          ),
          DeviceCommandDefinition(
              name="search_files",
              description="Search for files by name using Spotlight (mdfind).",
              input_schema={
                  "type": "object",
                  "properties": {
                      "query": {"type": "string", "description": "Filename or search term"},
                      "limit": {"type": "number", "description": "Max results (default 10)"},
                  },
                  "required": ["query"],
              },
          ),
          DeviceCommandDefinition(
              name="get_active_app",
              description="Get the name of the currently focused application.",
              input_schema={"type": "object", "properties": {}},
          ),
          DeviceCommandDefinition(
              name="system_info",
              description="Get system information: hostname, CPU, memory, uptime, and battery level.",
              input_schema={"type": "object", "properties": {}},
          ),
          DeviceCommandDefinition(
              name="get_clipboard",
              description="Read the current clipboard text contents.",
              input_schema={"type": "object", "properties": {}},
          ),
          DeviceCommandDefinition(
              name="set_clipboard",
              description="Set the clipboard text contents.",
              input_schema={
                  "type": "object",
                  "properties": {
                      "text": {"type": "string", "description": "Text to copy to clipboard"},
                  },
                  "required": ["text"],
              },
          ),
          DeviceCommandDefinition(
              name="lock_screen",
              description="Lock the Mac screen immediately.",
              input_schema={"type": "object", "properties": {}},
          ),
          DeviceCommandDefinition(
              name="play_music",
              description="Control music playback: play, pause, or skip to next/previous track.",
              input_schema={
                  "type": "object",
                  "properties": {
                      "action": {
                          "type": "string",
                          "enum": ["play", "pause", "next", "previous"],
                          "description": "Playback action",
                      },
                      "app": {
                          "type": "string",
                          "enum": ["Music", "Spotify"],
                          "description": "Music app to control (default: Music)",
                      },
                  },
                  "required": ["action"],
              },
          ),
          DeviceCommandDefinition(
              name="set_volume",
              description="Set the system output volume.",
              input_schema={
                  "type": "object",
                  "properties": {
                      "level": {"type": "number", "minimum": 0, "maximum": 100, "description": "Volume level 0-100"},
                  },
                  "required": ["level"],
              },
          ),
          DeviceCommandDefinition(
              name="run_shortcut",
              description="Run a macOS Shortcuts.app shortcut by name.",
              input_schema={
                  "type": "object",
                  "properties": {
                      "name": {"type": "string", "description": "Shortcut name"},
                  },
                  "required": ["name"],
              },
          ),
      ],
  )


  @client.on_command("take_screenshot")
  async def handle_take_screenshot(payload):
      filepath = os.path.join(tempfile.gettempdir(), f"screenshot-{int(datetime.now().timestamp())}.png")
      subprocess.run(["screencapture", "-x", filepath], check=True)
      with open(filepath, "rb") as f:
          buffer = f.read()
      url = await client.upload_file(buffer, f"screenshot-{int(datetime.now().timestamp())}.png", "image/png")
      os.unlink(filepath)
      return {"imageUrl": url, "timestamp": datetime.now(timezone.utc).isoformat()}


  @client.on_command("send_notification")
  async def handle_send_notification(payload):
      subprocess.run(
          ["terminal-notifier", "-title", payload["title"], "-message", payload["message"]],
          check=True,
      )
      return {"sent": True, "title": payload["title"], "message": payload["message"]}


  @client.on_command("open_url")
  async def handle_open_url(payload):
      subprocess.run(["open", payload["url"]], check=True)
      return {"opened": True, "url": payload["url"]}


  @client.on_command("open_app")
  async def handle_open_app(payload):
      subprocess.run(["open", "-a", payload["name"]], check=True)
      return {"opened": True, "app": payload["name"]}


  @client.on_command("search_files")
  async def handle_search_files(payload):
      limit = payload.get("limit", 10)
      result = subprocess.run(
          ["mdfind", "-name", payload["query"]],
          capture_output=True, text=True,
      )
      files = [f for f in result.stdout.strip().split("\n") if f][:limit]
      return {"query": payload["query"], "results": files, "count": len(files)}


  @client.on_command("get_active_app")
  async def handle_get_active_app(payload):
      script = 'tell application "System Events" to get name of first application process whose frontmost is true'
      result = subprocess.run(["osascript", "-e", script], capture_output=True, text=True)
      return {"activeApp": result.stdout.strip()}


  @client.on_command("system_info")
  async def handle_system_info(payload):
      hostname = subprocess.run(["hostname"], capture_output=True, text=True).stdout.strip()
      cpu = subprocess.run(["sysctl", "-n", "machdep.cpu.brand_string"], capture_output=True, text=True).stdout.strip()
      mem_bytes = int(subprocess.run(["sysctl", "-n", "hw.memsize"], capture_output=True, text=True).stdout.strip())
      mem_gb = round(mem_bytes / 1073741824)
      uptime = subprocess.run(["uptime"], capture_output=True, text=True).stdout.strip()
      try:
          batt = subprocess.run("pmset -g batt | grep -o '[0-9]*%'", shell=True, capture_output=True, text=True).stdout.strip()
      except Exception:
          batt = "N/A"
      return {"hostname": hostname, "cpu": cpu, "memoryGB": mem_gb, "uptime": uptime, "battery": batt or "N/A"}


  @client.on_command("get_clipboard")
  async def handle_get_clipboard(payload):
      result = subprocess.run(["osascript", "-e", "the clipboard"], capture_output=True, text=True)
      return {"clipboard": result.stdout.strip()}


  @client.on_command("set_clipboard")
  async def handle_set_clipboard(payload):
      escaped = payload["text"].replace('"', '\\"')
      subprocess.run(["osascript", "-e", f'set the clipboard to "{escaped}"'], check=True)
      return {"set": True, "text": payload["text"]}


  @client.on_command("lock_screen")
  async def handle_lock_screen(payload):
      subprocess.run(["pmset", "displaysleepnow"], check=True)
      return {"locked": True, "timestamp": datetime.now(timezone.utc).isoformat()}


  @client.on_command("play_music")
  async def handle_play_music(payload):
      app = payload.get("app", "Music")
      action_map = {"play": "play", "pause": "pause", "next": "next track", "previous": "previous track"}
      command = action_map[payload["action"]]
      subprocess.run(["osascript", "-e", f'tell application "{app}" to {command}'], check=True)
      return {"action": payload["action"], "app": app}


  @client.on_command("set_volume")
  async def handle_set_volume(payload):
      subprocess.run(["osascript", "-e", f'set volume output volume {payload["level"]}'], check=True)
      return {"volume": payload["level"]}


  @client.on_command("run_shortcut")
  async def handle_run_shortcut(payload):
      subprocess.run(["shortcuts", "run", payload["name"]], check=True)
      return {"ran": True, "shortcut": payload["name"]}


  async def main():
      await client.connect()
      print("Mac controller online")

  asyncio.run(main())
  ```
</CodeGroup>

## Agent Configuration

```typescript theme={null}
// src/index.ts
import { LuaAgent, LuaSkill } from 'lua-cli';

const macControlSkill = new LuaSkill({
  name: 'mac-controller',
  description: 'Control a MacBook remotely',
  context: `
    You are a personal assistant that controls a MacBook.
    You can take screenshots, open apps, search files, control music,
    manage clipboard, and more.

    Device tools:
    - take_screenshot: Captures the screen and returns an image URL.
    - send_notification: Shows a desktop notification with a title and message.
    - open_url: Opens a URL in the default browser.
    - open_app: Launches a Mac application by name.
    - search_files: Searches for files using Spotlight. Great for finding documents.
    - get_active_app: Reports which application is currently in the foreground.
    - system_info: Returns hostname, CPU, memory, uptime, and battery status.
    - get_clipboard: Reads the current clipboard text.
    - set_clipboard: Sets the clipboard to the provided text.
    - lock_screen: Locks the Mac immediately.
    - play_music: Controls Apple Music or Spotify (play, pause, next, previous).
    - set_volume: Sets the system volume from 0 to 100.
    - run_shortcut: Runs a Shortcuts.app shortcut by name.

    Guidelines:
    - When asked for a screenshot, take it and share the image URL
    - For file searches, show the full paths in the results
    - Confirm destructive actions (like locking the screen) before executing
    - Be conversational and helpful
  `,
  tools: [],
});

export const agent = new LuaAgent({
  name: 'mac-assistant',
  persona: `You are a personal assistant that controls a MacBook. You can take
    screenshots, open apps, search files, control music, manage the clipboard,
    and automate tasks. Be helpful, concise, and confirm before taking
    potentially disruptive actions like locking the screen.`,
  skills: [macControlSkill],
});
```

## What You Can Ask

Here are real conversational examples you can send from WhatsApp or web chat:

<CardGroup cols={2}>
  <Card title="Screenshots" icon="camera">
    "Take a screenshot and send it to me"
  </Card>

  <Card title="Open Apps" icon="window-maximize">
    "Open Slack" / "Launch Safari"
  </Card>

  <Card title="File Search" icon="magnifying-glass">
    "Search for files called invoice.pdf"
  </Card>

  <Card title="Active App" icon="desktop">
    "What app am I currently using?"
  </Card>

  <Card title="Lock Screen" icon="lock">
    "Lock my Mac"
  </Card>

  <Card title="Volume" icon="volume-high">
    "Set the volume to 30%"
  </Card>

  <Card title="Music" icon="music">
    "Play the next song" / "Pause the music"
  </Card>

  <Card title="Clipboard" icon="clipboard">
    "Copy this text to my clipboard: Meeting at 3pm"
  </Card>

  <Card title="System Info" icon="microchip">
    "How much battery do I have left?"
  </Card>

  <Card title="Notifications" icon="bell">
    "Send me a notification that says Stand up and stretch"
  </Card>

  <Card title="Browser" icon="globe">
    "Open github.com in my browser"
  </Card>

  <Card title="Shortcuts" icon="bolt">
    "Run my Focus Mode shortcut"
  </Card>
</CardGroup>

<Warning>
  **Security note:** This device client gives your AI agent direct control over your computer. Only run it on machines you trust, and consider limiting which commands are registered based on your comfort level.
</Warning>

## Next Steps

<CardGroup cols={2}>
  <Card title="Windows Controller" icon="windows" href="/devices/examples/windows-controller">
    Control a Windows PC the same way
  </Card>

  <Card title="Smart Office" icon="building" href="/devices/examples/smart-office">
    Office automation with sensors and displays
  </Card>

  <Card title="CDN Uploads" icon="cloud-arrow-up" href="/devices/cdn-uploads">
    How screenshot uploads work
  </Card>

  <Card title="Self-Describing Commands" icon="list-check" href="/devices/self-describing-commands">
    How to write effective command definitions
  </Card>
</CardGroup>