Turning market intelligence into a multi-CRM enrichment platform.
A two-year engineering partnership that took a B2B SaaS client's opportunity-intelligence data from analytics reports hitting hard sync limits to a self-serve, credit-metered enrichment engine running inside HubSpot, Zoho CRM, Salesforce and Freshsales.
- Client
- Confidential (B2B SaaS)
- Industry
- Sales intelligence
- Role
- Zoho & integrations developer
- Engagement
- Jun 2024 → ongoing
Opportunity intelligence is only useful where sales teams actually work.
The client runs an opportunity-intelligence platform for B2B software vendors. It tracks real buying signals — active trials, competitive churn, renewal windows and purchase intent — across tens of millions of companies, so go-to-market teams can focus on accounts that are genuinely in-market.
That intelligence lives in the client's own data warehouse. Its customers, however, live in their CRMs. The mission: build the middleware that continuously delivers each customer's licensed slice of that data directly into that customer's own CRM instance — reliably, at scale, and eventually with zero manual onboarding.
The data was rich. Getting it into customers' hands was the hard part.
Analytics reports timing out
High-volume Zoho Analytics reports failed with processing timeouts. Early syncs had to be re-architected onto query tables just to run at all.
Hard 25K-per-run sync limits
Native bulk-write sync capped at 25,000 records per run and burned 500 API credits per call — while customer datasets ran into the hundreds of thousands.
No product-owned integration
Early tests piggybacked on an off-the-shelf third-party sync app. The client needed its own app — private first, then marketplace-ready.
Every CRM speaks differently
Custom-field APIs differ per platform: ms_traffic in Zoho and HubSpot, cf_ms_traffic in Freshsales, opaque hash keys in Pipedrive.
Licensing and entitlement control
Customer A must never receive vendor data licensed to Customer B — and usage had to be metered with per-customer credits, which unified-API vendors couldn't control.
Query cost at scale
Serving the API and Chrome extension straight from the warehouse scanned gigabytes per request. Production lookups needed an indexed, low-latency store.
What success had to look like
Meet customers in their CRM
Deliver the client's intelligence natively into each customer's HubSpot, Zoho CRM, Salesforce or Freshsales instance — no exports, no spreadsheets.
Break the volume ceiling
Move past 25K-per-run platform limits to full-database syncs of hundreds of thousands of records, on a schedule.
Enforce entitlements and credits
Scope every payload to the vendors a customer has licensed, and meter consumption with a credit system the client fully owns.
Self-serve onboarding
Let a new customer connect their CRM, map fields and start their first sync through a branded activation portal — onboarding or removal, fully automatic.
A credit-metered enrichment engine between the warehouse and every CRM.
Rather than bending a low-code sync tool past its limits, the answer was a purpose-built Node.js middleware layer. Each CRM gets a dedicated sync service sharing the same core pattern: OAuth into the customer's instance, stream records out in pages, match them by domain against the client's PostgreSQL intelligence store, filter the payload down to the customer's licensed vendors, write enrichments back in batches — and burn credits for exactly what was processed.
The stack evolved deliberately: analytics query tables → scheduled CRM syncs → a VPS-hosted Node.js engine for 320K-record jobs → a warehouse-to-PostgreSQL migration for fast production queries. Unified-API platforms were evaluated hands-on for the multi-CRM layer, but per-customer credit control and predictable field mapping ultimately favoured direct, first-party integrations.
The whole story in two minutes.
Infitar IT Solutionsrecords synced into 4 CRMs.
Here's how we built it.
A B2B SaaS opportunity-intelligence platform
It tracks real buying signals — active trials, competitive churn, renewal windows, purchase intent — across millions of companies.
The catch? All that intelligence lived in a data warehouse.
Their customers live in their CRMs.
Native sync tools hit a wall
Reports timed out on volume. Bulk writes burned 500 API credits per call.
And the early integration ran on a third-party app the client didn't even own.
Stop renting. Build the engine.
A purpose-built Node.js middleware layer — owned end to end,
designed around every platform limit we'd mapped.
One engine, every CRM
Paginated OAuth fetch → domain-match against PostgreSQL → licensed-vendor filtering → batch write-back → credit burn → status webhook.
Built in, not bolted on
From capped to unstoppable
Delivered where sales teams work
Next up: Microsoft Dynamics 365.
Need your data
inside your CRM?
Zoho · HubSpot · Salesforce · Freshsales · Custom APIs
Everything shipped, nothing imagined
Per-CRM sync services
Dedicated Node.js services for HubSpot, Zoho, Salesforce and Freshsales, with OAuth 2.0, automatic token refresh and in-flight-request-safe rotation.
Domain-based enrichment matching
Records matched on domain, website or email domain against the PostgreSQL intelligence store — handling multiple CRM records per company.
Vendor entitlement filtering
Signal strings parsed and filtered so each customer only receives Active, Churned, Trial and Intent data for the vendors they've licensed.
Delta and Full sync modes
Full runs for first loads; delta runs sync only records created in the last 24 hours or updated within the freshness window — saving API credits on both sides.
Credit metering system
Per-customer credit balances in PostgreSQL, burned per record processed, floor-clamped at zero, with usage tracked for billing.
Batch write-back and rate control
Enriched payloads pushed via batch-update APIs in pages of 100 with pacing delays to stay inside vendor rate limits.
Job telemetry to Zoho CRM
Every run reports running and success states — records fetched, records processed, module, timestamp — into the client's Zoho CRM back office.
Self-serve activation portal
A branded portal lets customers authenticate their CRM, map fields and kick off syncs without developer involvement.
Warehouse → PostgreSQL migration
Production reads moved to Cloud SQL, cutting per-request scan costs and powering the public API and Chrome extension.
12-field intelligence payload
Country, company, eCommerce platform, active and churned subscriptions and trials, employees, traffic, annual revenue, LinkedIn URL and purchase intent.
How the platform evolved
Not a big-bang build — a sequence of working systems, each one unlocking the next stage of scale.
Foundations and first sync
Rebuilt failing analytics reports on query tables, launched the first scheduled Zoho Analytics → Zoho CRM enrichment sync, and mapped the platform's real limits: 25K records per run, 500 credits per bulk write.
Product-owned HubSpot app
Designed the client's own middleware app — private first, marketplace next — to enrich client-specific HubSpot instances, replacing the borrowed off-the-shelf sync used for early testing.
VPS-powered scale engine
Moved heavy lifting to Node.js services on a Windows VPS. Validated a 320K-record sync end to end, orchestrated runs from Zoho CRM, and split production and development environments.
Data layer re-platform
Migrated production queries from warehouse table scans to PostgreSQL on Cloud SQL — indexed domain lookups for the sync engine, public API and Chrome extension.
Multi-CRM and monetization
Evaluated unified-API platforms hands-on, then extended first-party integrations to Freshsales and Salesforce, wired in per-customer credit metering, and launched self-serve activation.
Expansion and ongoing support
Microsoft Dynamics 365 integration underway, cloud migration support from GCP to Azure, and continuous production operations — monitoring, incident response and iteration.
The engineering under the hood
Two patterns from the HubSpot sync service illustrate the approach: long-running jobs must never die mid-flight, and licensed data must be filtered at the string level, not just the row level.
// Long syncs outlive OAuth tokens. Before each page, // check expiry (with a 2-min safety buffer) and pause // so in-flight calls finish before rotating the token. if (Date.now() >= accessHolder.expiresAt) { console.log("Token about to expire — waiting…"); await delay(10_000); accessHolder.token = await refreshAccessToken(rt); accessHolder.expiresAt = crmTokenExpiresAt; }
// Signal fields pack many vendors into one string. // Split on '//' and keep only segments whose vendor // name — plain or parenthetical — is licensed. const matched = segments.filter(seg => { const vendor = seg.slice(0, seg.indexOf(':')); return allowed.some(v => eq(vendor, v) || parenMatch(vendor, v)); }); return matched.join(' // ') || null;
IN queries, batch-updating enriched records, and finally burning credits equal to records processed before reporting fetched and processed counts back for the ops dashboard.From manual plumbing to a product
ø Before
- Enrichment tested through a third-party sync app the client didn't own
- Reports timing out on high-volume data; syncs capped at 25K records per run
- Developer-driven onboarding for every new customer connection
- No per-customer entitlement filtering or usage metering
- Production API reads scanning the full warehouse per request
✓ After
- Client-owned sync apps across HubSpot, Zoho CRM, Salesforce and Freshsales
- 320K-record syncs validated end to end on the VPS engine, with delta runs for freshness
- Self-serve connect → map → sync via a branded activation portal
- Vendor-scoped payloads plus a credit system powering usage-based pricing
- Indexed PostgreSQL lookups serving syncs, the public API and the Chrome extension
Measured where it could be, honest where it couldn't.
The clearest quantitative outcome is scale: single sync runs validated at 320K+ records against a platform-native ceiling of 25K — roughly a 12× jump in per-run capacity. Beyond that, the wins are structural: four live CRM integrations instead of one borrowed app, automated customer onboarding instead of developer-led setup, and a metering layer that turned raw data delivery into a billable, controllable product surface. The engagement itself is a result — over 21 months of continuous trust, with scope steadily widening from analytics fixes to core platform infrastructure.
In their own words
Arfater is a walking encyclopedia on all things integration and how to get data in and out across the Zoho stack.
Thank you for making it — such a great presenter you are! Excellent explanation and covered everything.
This is amazing progress, guys.
What this project taught
Platform limits are product requirements in disguise
The 25K-per-run cap wasn't a bug to complain about — it was the specification for the custom engine. Discovering hard limits early, and designing around them deliberately, shaped the whole architecture.
Buy versus build has to be tested, not assumed
Unified-API platforms promised instant multi-CRM reach, but hands-on trials exposed dealbreakers: no per-customer credit control and unpredictable custom-field naming. Direct integrations cost more effort and bought full ownership.
Every CRM's "same" API is different
Field-key conventions alone — ms_traffic versus cf_ms_traffic versus hashed keys — can make or break an automated mapping flow. Abstractions must be designed around the messiest platform, not the cleanest.
Long-running jobs need defensive plumbing
Token expiry mid-run, rate limits, IP changes, restarts — resilience patterns like expiry buffers, in-flight drain delays, batch pacing and status heartbeats are what make a pipeline production-grade.
Data licensing is an engineering problem
Entitlement enforcement had to live in code — string-level vendor filtering plus credit metering — before the client could safely scale a paid, multi-tenant data product.
Chosen for control, not convenience
Infrastructure that turned data into a distributable product.
What began as fixing a timing-out analytics report became the delivery backbone of the client's business: a first-party, credit-metered enrichment platform that puts opportunity intelligence inside the CRMs where B2B sales teams already work — and keeps growing, with Dynamics 365 and a new cloud environment next on the roadmap.