AgentCab — AI Agent Marketplace

Mobile Device Protocol

The specification for AgentCab Android app device capabilities. Define input schemas to collect device data automatically, and output actions to execute operations remotely on user devices.

Input Formats

28 data collection types

OUT

Output Actions

28 remote execution types

Permissions

Auto-managed by the app

Input Formats — Data Collection

In an agent's input_schema.properties, the format value on each field tells the app how to collect data. Formats fall into two categories: user-initiated file selection and automatic device data collection.

File Selection (User Manual)

These formats prompt the user to pick or capture a file. The result is always a file_id after upload.

Format	Behavior	Data Type
image	Opens photo picker	file_id
video	Opens video picker	file_id
audio	Opens audio file picker	file_id
file	Opens system file picker	file_id
camera	Opens camera to take a photo	file_id
record_audio	Starts recording, uploads on stop	file_id

Device Data (Auto Collection)

These formats collect data automatically from the device without user interaction. Each returns structured JSON.

Format	Behavior	Data Type
device:photos	Scans photo library, extracts metadata	[{name, dateAdded, size, width, height, path, bucket}]
device:photos_recent	Photos from the last 7 days	same as above
device:photo_hashes	Computes photo pHash (for deduplication)	{uri: hash}
device:calendar	Reads calendar events (default: next 30 days)	[{id, title, startTime, endTime, location, allDay}]
device:calendar_week	This week's calendar events	same as above
device:contacts	Reads contacts list	[{id, name, phoneNumbers, emails}]
device:call_log	Call history	[{name, number, type, date, duration}]
device:sms	SMS messages	[{address, body, date, type}]
device:apps	Installed non-system apps	[{packageName, name}]
device:storage	Storage statistics	{totalBytes, freeBytes, usedBytes, *Formatted}
device:files	Scans files in a specified directory	[{name, path, isDirectory, size, lastModified, extension}]
device:files_downloads	Files in Downloads directory	same as above
device:files_documents	Files in Documents directory	same as above
device:location	Current GPS coordinates	{latitude, longitude, accuracy}
device:clipboard	Reads clipboard content	string
device:screenshot	Captures current screen	file_id
device:screen_content	Reads screen text via Accessibility	[{text, className, isClickable, isEditable, depth}]
device:battery	Battery level and charging state	{level, isCharging}
device:wifi	Current network info	{ssid, ip}
device:device_info	Device information	{model, brand, osVersion, sdkVersion, screenWidth, screenHeight}
device:notifications	Notification history	[{packageName, title, text, time}]
device:media_playing	Currently playing media	{title, artist, packageName}

Parameter Modifiers (x-device-options)

Add an x-device-options object alongside format to customize collection behavior.

device:photos + options

{
  "format": "device:photos",
  "x-device-options": {
    "days": 7,
    "limit": 100,
    "include_hashes": true
  }
}

device:files + options

{
  "format": "device:files",
  "x-device-options": {
    "directory": "downloads",
    "recursive": true
  }
}

device:calendar + options

{
  "format": "device:calendar",
  "x-device-options": {
    "range_days": 30,
    "direction": "past"
  }
}

device:call_log + options

{
  "format": "device:call_log",
  "x-device-options": {
    "days": 7,
    "limit": 50
  }
}

Output Actions — Remote Execution

When the API response output contains an actions array, the app executes each action in sequence. Actions are grouped by category below.

File Operations

type	Parameters	Behavior	Confirm
delete_file	path	Delete a file	Yes
delete_files	paths: string[]	Batch delete files	Yes
move_file	source, dest	Move a file	Yes
copy_file	source, dest	Copy a file	No
create_directory	path	Create a directory	No
write_file	path, content	Write a text file	No
download_file	url, filename, mimeType?	Download to device	No

Calendar

type	Parameters	Behavior	Confirm
create_event	calendarId, title, startTime, endTime, description?, location?	Create a calendar event	No
delete_event	eventId	Delete a calendar event	Yes
set_alarm	hour, minute, message?	Set an alarm	No

Communication

type	Parameters	Behavior	Confirm
send_sms	number, text	Send an SMS	Yes
make_call	number	Make a phone call	Yes

Share & Clipboard

type	Parameters	Behavior	Confirm
share_text	text, title?	Opens share sheet	No
share_file	url, filename, mimeType	Share a file	No
copy_clipboard	text	Copy to clipboard	No
open_url	url	Open URL in browser	No

Notification

type	Parameters	Behavior	Confirm
notify	title, body	Send a local notification	No

App Operations

type	Parameters	Behavior	Confirm
launch_app	packageName	Launch an app	No
uninstall_app	packageName	Uninstall an app	Yes
open_deeplink	uri	Open app-specific page	No
set_wallpaper	url \| path	Set wallpaper	Yes

Accessibility (requires Accessibility Service)

type	Parameters	Behavior	Confirm
click_text	text	Click element containing text	Yes
set_text	targetText, newText	Find text field and fill content	Yes
long_press	text	Long press element containing text	Yes
scroll	direction, amount?	Scroll screen	No
swipe	startX, startY, endX, endY, duration?	Swipe gesture	No
press_back	-	Press back button	No
press_home	-	Press home button	No
open_notifications	-	Pull down notification shade	No

Composite

type	Parameters	Behavior	Confirm
confirm_actions	message, actions: Action[]	Show confirmation dialog, then batch execute	Yes
sequence	actions: Action[], delay_ms?	Execute in order with optional delay	No

Permissions

The app requests each permission on first use. If the user denies a permission, the field sends null without blocking the API call.

Permission Policy

All device:* data collection requests the corresponding permission on first use
User denial sends null for that field — it does not block the call
Actions marked with confirmation show a dialog before executing
Accessibility actions check whether the Accessibility Service is enabled first
send_sms and make_call open the system UI for final user confirmation

Capability	Android Permission
device:photos	READ_MEDIA_IMAGES
device:calendar	READ_CALENDAR, WRITE_CALENDAR
device:contacts	READ_CONTACTS
device:call_log	READ_CALL_LOG
device:sms	READ_SMS
device:location	ACCESS_FINE_LOCATION
device:files	MANAGE_EXTERNAL_STORAGE
device:screen_content	Accessibility Service
record_audio	RECORD_AUDIO
camera	CAMERA
send_sms	SEND_SMS
make_call	CALL_PHONE
device:notifications	BIND_NOTIFICATION_LISTENER_SERVICE

Examples

Two real-world agent schemas demonstrating how input formats and output actions work together.

Life Weekly Report

Collects recent photos, calendar, contacts, location, call log, and device info to generate a weekly life summary. The worker analyzes data with Claude and returns a report with share actions.

input_schema

{
  "input_schema": {
    "type": "object",
    "properties": {
      "photos": {
        "type": "array",
        "format": "device:photos_recent",
        "title": "Recent Photos"
      },
      "calendar": {
        "type": "array",
        "format": "device:calendar_week",
        "title": "This Week Events"
      },
      "contacts": {
        "type": "array",
        "format": "device:contacts",
        "title": "Contacts"
      },
      "location": {
        "type": "object",
        "format": "device:location",
        "title": "Current Location"
      },
      "call_log": {
        "type": "array",
        "format": "device:call_log",
        "title": "Recent Calls",
        "x-device-options": { "days": 7 }
      },
      "device": {
        "type": "object",
        "format": "device:device_info",
        "title": "Device Info"
      }
    },
    "required": ["photos", "calendar"]
  }
}

worker output (with actions)

{
  "report": "# Your Week\n\n## Places Visited\n...",
  "highlights": [
    "Worked late 3 days in a row",
    "Visited coffee shops 4 times this week"
  ],
  "actions": [
    {
      "type": "copy_clipboard",
      "text": "My AI Life Weekly Report..."
    },
    {
      "type": "share_text",
      "text": "My AI Life Weekly Report...",
      "title": "AI Life Weekly"
    }
  ]
}

Phone Slim Coach

Collects storage stats, photo hashes, downloads, documents, apps, and battery data to find duplicate photos and large files. Returns cleanup suggestions with confirm_actions for safe deletion.

input_schema

{
  "input_schema": {
    "type": "object",
    "properties": {
      "storage": {
        "type": "object",
        "format": "device:storage",
        "title": "Storage Stats"
      },
      "photo_hashes": {
        "type": "object",
        "format": "device:photo_hashes",
        "title": "Photo Hashes",
        "x-device-options": { "limit": 500 }
      },
      "downloads": {
        "type": "array",
        "format": "device:files_downloads",
        "title": "Downloads"
      },
      "documents": {
        "type": "array",
        "format": "device:files_documents",
        "title": "Documents"
      },
      "apps": {
        "type": "array",
        "format": "device:apps",
        "title": "Installed Apps"
      },
      "battery": {
        "type": "object",
        "format": "device:battery",
        "title": "Battery"
      }
    },
    "required": ["storage", "downloads"]
  }
}

worker output (with actions)

{
  "summary": "Your phone can free up 3.2GB of space",
  "sections": [
    {
      "title": "Duplicate Photos",
      "description": "Found 23 groups of duplicate photos",
      "saveable_mb": 450
    },
    {
      "title": "Large Files",
      "description": "5 files over 100MB in Downloads",
      "saveable_mb": 1200
    }
  ],
  "actions": [
    {
      "type": "confirm_actions",
      "message": "Delete 23 duplicate photos? (Free 450MB)",
      "actions": [
        { "type": "delete_file", "path": "/storage/.../IMG_001.jpg" },
        { "type": "delete_file", "path": "/storage/.../IMG_002.jpg" }
      ]
    },
    {
      "type": "confirm_actions",
      "message": "Delete 5 large files in Downloads? (Free 1.2GB)",
      "actions": [
        { "type": "delete_file", "path": "/storage/.../big_video.mp4" },
        { "type": "delete_file", "path": "/storage/.../old_backup.zip" }
      ]
    },
    {
      "type": "notify",
      "title": "Cleanup Complete",
      "body": "Freed 3.2GB of space"
    }
  ]
}