Control a laptop from chat

You’ll build a device that turns your own laptop into something the agent can operate from any channel: “take a screenshot”, “open Notes”, “lock my screen”. The finished script is about 80 lines, runs with npx tsx device.ts, and passes everything the model says to the operating system as arguments, never inside a shell command. You need Node 18 or later, an agent created with lua init, and on macOS the Screen Recording permission for your terminal. Verified against lua-cli 3.45.0.

What you’ll build

A device named my-laptop with six commands the model can call.
Screenshots uploaded to the CDN so the agent can share an image URL.
An allow-list for the applications the agent may open.
A notification, volume, and lock-screen command driven by sanitized arguments.

Create the credential

From the agent project, issue a device credential for device name my-laptop with the operations commands and assets.upload, written to the folder the device will run from. Then load it there and install the client.

mkdir -p ../my-laptop
lua devices credential --device-name my-laptop --operations commands,assets.upload --out ../my-laptop/.env.device
cd ../my-laptop
set -a; . ./.env.device; set +a
npm init -y && npm install @lua-ai-global/device-client && npm install -D tsx

Write the device

Model output reaches the OS only as execFileSync arguments or environment variables, and open_app checks an allow-list first.

macOS
Windows

device.ts

import { execFileSync } from 'node:child_process';
import { readFileSync, unlinkSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { DeviceClient } from '@lua-ai-global/device-client';

const ALLOWED_APPS = ['Safari', 'Notes', 'Music', 'Terminal'];

const device = new DeviceClient({
  agentId: process.env.LUA_AGENT_ID!,
  deviceCredential: process.env.LUA_DEVICE_CREDENTIAL!,
  deviceName: 'my-laptop',
  commands: [
    { name: 'take_screenshot', description: 'Capture the whole screen and return an image URL', timeoutMs: 15000 },
    {
      name: 'open_app',
      description: `Open one of these applications: ${ALLOWED_APPS.join(', ')}`,
      inputSchema: { type: 'object', properties: { name: { type: 'string', enum: ALLOWED_APPS } }, required: ['name'] },
    },
    {
      name: 'set_volume',
      description: 'Set the output volume, 0 to 100',
      inputSchema: { type: 'object', properties: { level: { type: 'integer', minimum: 0, maximum: 100 } }, required: ['level'] },
    },
    { name: 'get_active_app', description: 'Name the application in the foreground' },
    {
      name: 'send_notification',
      description: 'Show a desktop notification',
      inputSchema: { type: 'object', properties: { title: { type: 'string' }, message: { type: 'string' } }, required: ['title', 'message'] },
    },
    { name: 'lock_screen', description: 'Sleep the display, which locks the screen when a wake password is set' },
  ],
});

// AppleScript reads model text from the environment, never from the script source.
function osascript(script: string, env: Record<string, string> = {}): string {
  return execFileSync('osascript', ['-e', script], { encoding: 'utf8', env: { ...process.env, ...env } }).trim();
}

device.onCommand('take_screenshot', async () => {
  const path = join(tmpdir(), `screenshot-${Date.now()}.png`);
  execFileSync('screencapture', ['-x', path]);
  const file = await device.cdn.upload(readFileSync(path), 'screenshot.png', 'image/png');
  unlinkSync(path);
  return { url: file.url };
});

device.onCommand('open_app', async (payload) => {
  const name = String(payload.name);
  if (!ALLOWED_APPS.includes(name)) throw new Error(`${name} is not on the allow-list`);
  execFileSync('open', ['-a', name]);
  return { opened: name };
});

device.onCommand('set_volume', async (payload) => {
  const level = Math.round(Number(payload.level));
  if (!Number.isFinite(level) || level < 0 || level > 100) throw new Error('level must be 0 to 100');
  osascript(`set volume output volume ${level}`);
  return { volume: level };
});

device.onCommand('get_active_app', async () => ({
  app: osascript('tell application "System Events" to get name of first application process whose frontmost is true'),
}));

device.onCommand('send_notification', async (payload) => {
  osascript('display notification (system attribute "LUA_MESSAGE") with title (system attribute "LUA_TITLE")', {
    LUA_TITLE: String(payload.title),
    LUA_MESSAGE: String(payload.message),
  });
  return { shown: true };
});

device.onCommand('lock_screen', async () => {
  execFileSync('pmset', ['displaysleepnow']);
  return { locked: true };
});

device.on('connected', () => console.log('my-laptop connected'));
device.connect().catch((err: Error) => {
  console.error(err.message);
  process.exit(1);
});

device.ts

import { execFileSync } from 'node:child_process';
import { readFileSync, unlinkSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { DeviceClient } from '@lua-ai-global/device-client';

const ALLOWED_APPS = ['notepad', 'calc', 'msedge'];

const device = new DeviceClient({
  agentId: process.env.LUA_AGENT_ID!,
  deviceCredential: process.env.LUA_DEVICE_CREDENTIAL!,
  deviceName: 'my-laptop',
  commands: [
    { name: 'take_screenshot', description: 'Capture the primary screen and return an image URL', timeoutMs: 15000 },
    {
      name: 'open_app',
      description: `Open one of these applications: ${ALLOWED_APPS.join(', ')}`,
      inputSchema: { type: 'object', properties: { name: { type: 'string', enum: ALLOWED_APPS } }, required: ['name'] },
    },
    { name: 'get_active_window', description: 'Return the title of the foreground window' },
    { name: 'lock_screen', description: 'Lock the screen' },
  ],
});

// The PowerShell source is constant; model text arrives through the environment.
function powershell(script: string, env: Record<string, string> = {}): string {
  return execFileSync('powershell', ['-NoProfile', '-NonInteractive', '-Command', script], {
    encoding: 'utf8',
    env: { ...process.env, ...env },
  }).trim();
}

device.onCommand('take_screenshot', async () => {
  const path = join(tmpdir(), `screenshot-${Date.now()}.png`);
  powershell(
    [
      'Add-Type -AssemblyName System.Windows.Forms,System.Drawing',
      '$b = [System.Windows.Forms.Screen]::PrimaryScreen.Bounds',
      '$bmp = New-Object System.Drawing.Bitmap $b.Width, $b.Height',
      '$g = [System.Drawing.Graphics]::FromImage($bmp)',
      '$g.CopyFromScreen($b.Location, [System.Drawing.Point]::Empty, $b.Size)',
      '$bmp.Save($env:LUA_OUT, [System.Drawing.Imaging.ImageFormat]::Png)',
    ].join('; '),
    { LUA_OUT: path }
  );
  const file = await device.cdn.upload(readFileSync(path), 'screenshot.png', 'image/png');
  unlinkSync(path);
  return { url: file.url };
});

device.onCommand('open_app', async (payload) => {
  const name = String(payload.name);
  if (!ALLOWED_APPS.includes(name)) throw new Error(`${name} is not on the allow-list`);
  powershell('Start-Process -FilePath $env:LUA_APP', { LUA_APP: name });
  return { opened: name };
});

device.onCommand('get_active_window', async () => ({
  title: powershell(
    [
      'Add-Type -Namespace W -Name U -MemberDefinition \'[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow(); [DllImport("user32.dll")] public static extern int GetWindowText(IntPtr h, System.Text.StringBuilder s, int n);\'',
      '$sb = New-Object System.Text.StringBuilder 512',
      '[void][W.U]::GetWindowText([W.U]::GetForegroundWindow(), $sb, 512)',
      '$sb.ToString()',
    ].join('; ')
  ),
}));

device.onCommand('lock_screen', async () => {
  execFileSync('rundll32.exe', ['user32.dll,LockWorkStation']);
  return { locked: true };
});

device.on('connected', () => console.log('my-laptop connected'));
device.connect().catch((err: Error) => {
  console.error(err.message);
  process.exit(1);
});

Run the device

Start the script and leave it running. It prints my-laptop connected; on macOS the first screenshot prompts for the Screen Recording permission.

npx tsx device.ts

Ask the agent

From the agent project, try each command. The model calls device__my_laptop__take_screenshot and replies with the image URL; open_app refuses anything outside the allow-list and the model reports the error.

lua chat -m "Take a screenshot of my laptop and send me the link"
lua chat -m "Open Notes on my laptop"

Every command runs on your machine whenever the agent calls it, from any channel the agent is connected to. Keep the allow-list short and stop the script when you’re done.

Verify

Check the device state from the agent project.

lua devices status --device-name my-laptop --ci

It prints 🟢 Device 'my-laptop' is online; stop the script and it prints offline and the tools disappear from the agent.

What you learned

A device describes its commands at connect time; no agent code changed. See Expose device commands as tools.
inputSchema with enum plus a check in the handler keeps model output out of shell strings.
device.cdn.upload needs the assets.upload operation. See Upload files from a device.
Tools exist only while the device is online. See How devices work.

Get started

Concepts

Build

Channels

Integrations

Devices

Ship and operate

Ship logs to your stack

Marketplace

Build with AI

Examples

Control a laptop from chat

What you’ll build

What you learned

Next steps

Handle device triggers

Node device client

​What you’ll build

​What you learned

​Next steps

Handle device triggers

Node device client

What you’ll build

What you learned

Next steps