Data Connectors

Ema grounds every answer in reliable, current information. Data connectors make this possible by synchronizing internal files -- tickets, policies, wikis, spreadsheets -- and, when needed, carefully selected public-web pages. Together they create a single, continuously refreshed knowledge base that every AI Employee can cite with confidence.

File Ingestion

Ema offers three options for bringing enterprise documents into the platform. Begin with one and expand as your needs grow.

How Ema Brings Data In

• Local upload -- Drag files onto the Canvas and validate parsing in seconds.
• Cloud-storage sync -- Connect Google Drive, SharePoint, OneDrive, Box, or Dropbox. Ema keeps its data updated with the linked storage provider.
• Knowledge-base APIs -- Link Confluence, ServiceNow, and similar platforms to keep wikis and runbooks fresh. The maximum per-file size supported for this ingestion type is 200 MB.

All three paths feed a shared pipeline, so a bulk PDF import today and a SharePoint sync tomorrow do not require re-indexing.

Knowledge Base APIs must be connected and authenticated from the Integrations page. Once connected, these connectors are available only from the Shared Configuration option. They are not available from specialized AI Employees like Recruiter or Document Generation.

Knowledge Base Connector Scope

Ema connects at the knowledge base level (or its equivalent in each platform), not at the instance or tenant level. When you connect a knowledge base, Ema ingests all content within it -- articles, folders, attachments, and nested categories -- automatically.

Instance-level connections are not supported because they bypass access controls and segmentation, and introduce irrelevant content that degrades AI response quality.

Terminology Across Connectors

If you need content from multiple knowledge bases or spaces, connect each one individually through the Integrations page.

File-Format Support

Scanned documents pass through OCR, but diagrams are not interpreted. Add captions when images matter. OCR capabilities are actively being improved.

Preparing for a Connector Roll-out

1. Confirm every file is in a supported format.
2. Set up OAuth scopes or service-account keys with read-only access.
3. Test ten representative files to verify parsing quality and latency.
4. Estimate daily volume and adjust concurrency settings to prevent large syncs from interfering with other operations.

What to Expect at Runtime

• Markdown, HTML, and CSV files ingest almost instantly.
• Office files and text-only PDFs take a few seconds per megabyte.
• Image-heavy PDFs and media-rich presentations run three to four times slower.
• Fully scanned documents require an additional OCR pass.

Security Fundamentals

The security architecture integrates OAuth 2.0 for cloud sources and service-account credentials for on-premises systems. All webhook callbacks are digitally signed. Connector dashboards clearly display rate limits, enabling you to prevent potential overloads.

Known Limitations and Mitigations

• XLSB files are partially supported. Consider exporting to XLSX format until native parsing becomes available.
• Very long PDFs with complex tables may flatten. Consider splitting the PDF or converting tables to CSV format.
• No image-semantic extraction. Include explanatory text adjacent to screenshots.

Web Content Scraper

Some business questions draw on information outside corporate systems -- partner portals, regulator updates, industry resources. The web content scraper captures this public content and folds it into the same index that serves internal documents.

Purpose and Fit

Common use cases where web scraping provides critical business value:

• Regulatory tracking -- Compliance teams reference new agency guidance without manual copy-paste.
• Supply-chain status -- Operations staff rely on carrier or supplier pages to predict delays.
• Investor messaging -- Finance groups pull public FAQs and statements to prepare earnings-call answers.

How the Scraper Works

Supported and Unsupported Sites

Implementation Guidance

• Seed precisely with targeted URLs (for example, /support/faq) rather than the main website homepage to keep crawl times predictable.
• Stage and validate using a short list first; inspect parsed output in the dashboard, then widen scope.
• Plan runtime because a two-level crawl can take several minutes; allocate processing windows accordingly.
• Re-crawl when needed by submitting the URL list again whenever the source site posts significant updates.

Knowledge Base API

Ema fetches each URL you list from your API, indexes the response as one document, and keeps it in sync. One URL, one document. Ema does not crawl or follow links.

How It Works

Base URL + list of paths → Ema fetches each URL → One document per URL, indexed → Searchable by the AI Employee

Before You Start

Supported content: JSON, HTML, Markdown, plain text, XML, CSV, PDF and Office documents. Limits: 2,000 documents per source, 10 MB per document.

Journey 1: Connect a New Source

Open the AI Employee, go to Configuration, open the folder, click Upload, then Connect on the Knowledge base API tile.

1. Authentication method. Public, API key, or OAuth 2.0. For API key, enter the header name and the key. For a bearer token, use Authorization and Bearer <token>. For OAuth 2.0, enter client ID, client secret, and token endpoint path.
2. Base URL. The host, nothing after it. https://kb.example.com, not https://kb.example.com/api/.
3. Document paths. Type them, or Import CSV. Download sample shows the format: one path per row in the first column, optional header row. Full URLs on the same host are accepted; other hosts are refused.
4. Advanced. Only if your API needs a header on every request, such as Accept-Language: en-US.
5. Connect. Ema fetches the first document with your credential. A green tick and "Fetching N documents into <host>" means it works. An error above the form means it does not; the source is not kept, fix the form and connect again.

Connect → Ema fetches first document → Works → fetch & index → Fails → fix & retry

Journey 2: Add Documents to a Connected Source

Each source is a tile named after its host. Click Add, enter paths or import a CSV, click Add documents. Paths already in the folder are skipped. No credential to enter again.

Journey 3: Keep Documents in Sync

Set Auto sync on the folder, or sync by hand. Unchanged documents are skipped, changed ones replaced. A URL that fails is marked Failed with the reason and retried next sync; the rest continue.

Reading Errors

The message is the source's own reason or Ema's description. Your credential never appears in it. On a failed Connect, the reason appears above the form.

Connector-Specific Guides

• Confluence Data Connector