Changelog

Configurable rate limits

One of the biggest reasons customers adopt Sequin is because they're hitting rate limits. Rate limits make every integration more complex. When your integration is new and small, you still have to consider rate limits – what if something in your system causes a burst of query activity? As you scale, rate limits can bottleneck your entire system. Pacing out requests isn't enough for large integrations. You need to cache your reads and batch your writes as well.

Sequin handles caching and batching for you. The data we sync to your database becomes your new source of truth, so you don't have to hit the API every time you want to fetch data. On the write side, we're rolling our sophisticated batching features.

You provide Sequin with a rate limit. Sequin uses that rate limit to pace the sync for the read side and pace out the writes you make through your database or our proxy.

Our rate limit was minimally configurable before, but starting today you have full control:

We'll be adding features to this rate limit configuration soon, including a calculator that will help you understand how different values for your rate limit affect your sync and write speed.

Fixes and improvements

• Removed the toggles for "Sync all current and future" tables and columns. While a nice convenience function, it was hard for customers to understand the behavior of these features. Adding new tables and columns requires migrations. Migrations are significant and migrations getting stuck due to things like view dependencies is the primary reason that syncs run into issues. We're moving to a world where we'll notify customers about changes to their API objects and fields. That way, you can opt-in to the changes and be there to intervene if something goes wrong.
• Fixed an issue with HubSpot batch deletes when a lot of records were deleted at once.
• Added support for *_hd Salesforce objects.

Bidirectional sync for HubSpot

Our HubSpot sync is now bidirectional. When you make changes in your database, they'll propagate back to HubSpot. With bidirectional sync, objects and fields in HubSpot are linked to tables and columns in your database: create a Deal in HubSpot and it will be synced to your database. Update the deal in your database and it will be synced back to HubSpot.

With bidirectional sync, you don't have to worry about the HubSpot rate limit, batching requests, query params, and keeping two sources of truth in-sync. Sequin's got it for you.

SSH tunnels for database connections

If you're syncing to a database that's located in a private VPC, Sequin can use a jump host to access it. During database setup, you'll see an option for "SSH Tunnel." Plug the hostname for your jump host in there and then add Sequin's public key to your server's authorized keys.

Fixes and improvements

• Sync performance: When we detect a particular endpoint is busy (lots of changes happening), we'll favor requests to that endpoint (while making fewer requests to dormant endpoints.)
• Console performance: Loading the column mapping when creating a sync took a long time for big Salesforce accounts. We've optimized this a lot, so each stage should load quickly.
• Fixed an issue with HubSpot OAuth that caused the flow to fail on the final leg.
• Fixes and improvements for both the Salesforce and HubSpot sync processes.
• Fixed some visual inconsistencies in the graph that displays your usage for the billing cycle.

Fixes and improvements

• Console performance: the console became very unwieldy when dealing with huge Salesforce instances (hundreds of tables each with hundreds of fields.) We've improved the performance in these instances dramatically.
• Coalescing writes: When multiple changes are made to a single row in quick succession, we coalesce those changes into one write request to the API, saving requests.
• General performance tuning and improvements for syncs.

Two-way sync alpha

Two-way sync has arrived for Salesforce! You can now make changes in your database and they'll propagate back to Salesforce. This means objects and fields in Salesforce are truly linked to tables and columns in your database: create a Contact in Salesforce, it's synced to your database. Update the contact in your database, and it's synced back to Salesforce.

With two-way sync, you don't have to worry about the Salesforce rate limit, batching requests, SOQL, and keeping two sources of truth in-sync. Sequin's got it for you.

Next up is HubSpot.

Fixes and improvements

• HubSpot: Add support for HubSpot pipelines and stages.
• HubSpot and Salesforce: Added support for polling for schema changes for HubSpot and Salesforce. Schema changes in those platforms now automatically propagate to the database without intervention.
• Stripe: Fixed an issue that caused us to miss some updates to payment intents.
• Fixed an issue where HubSpot and Salesforce syncs would not shutdown properly when deactivated.
• Improved how we handled situations where HubSpot returned bad responses, specifically returning objects out-of-order.

HubSpot alpha

HubSpot is now on Sequin! Sync your HubSpot Contacts and Deals right to your database. With Sequin, your sales and support teams can store and work with information about your customers over on HubSpot. And your developers can easily pull that data into your main application or tools.

More coming for HubSpot soon.

Fixes and improvements

• Fixed issues with performance in the console. The console now loads and responds much faster.
• Fixed issues with our Salesforce sync that would cause the sync to halt.

Column mapping

Column mapping introduces two key features Sequin customers have been asking for.

Renaming: Sequin does its best to take API object and field names and convert them to SQL-friendly table and column names. But sometimes, you want even more control. With renaming, you can now customize field names to best fit your SQL database.

Locking: With dynamic platforms like Salesforce or Airtable, schemas change all the time. A team member might change the name of an object or a field without alerting the rest of the team.

Without locking, Sequin will pick up those changes and propagate them to your database. This means any SQL or ORM code you've written will break, often without warning. With locking, you can have Sequin lock or freeze the column names as they are. The database schema will be protected from schema changes made to the schema in the upstream API (e.g. Salesforce or Airtable.) You can come back and update the column name to the new upstream name whenever you're ready.

Fixes and improvements

• SSH tunnels: Our tunnels to database had reliability issues. We fixed the underlying issue to make them more reliable and added an automatic recovery mechanism if they fail in the future.
• Improved the health status for syncs on the console. Now we factor in previous sync durations into diagnosis of a sync's health.
• Fixed an issue that would cause syncs with very large Airtable bases to intermittently start over.
• Fixed an issue with database setup. We now verify we have the right permissions in the database during setup. This got rid of an issue where customers could successfully add a database to Sequin, even though it was improperly configured.
• Improved security around password reset flow.
• Updated pricing page with a calculator that should help give better estimates on our pricing.

We're cooking up a lot of bigger updates that will be released over the next two weeks. Here are some of the smaller changes that were released in this cycle:

Fixes and improvements

• Console performance improvements. If you have a lot of syncs, you'll see the difference.
• Fixed an issue where users can't edit synced tables for Stripe.
• Fixed an issue with account activation after subscribing.
• Better handle bad or wrongly permissioned tokens from Salesforce.
• After creating a new sync, make it clear that we'll send an email when the backfill completes.

Salesforce alpha

We're live with Salesforce! Sync your Salesforce Contacts and Leads right to your database. With Sequin, your sales and support teams can store and work with information about your customers over on Salesforce. And your developers can easily pull that data into your main application or tools.

More coming for Salesforce soon.

Fixes and improvements

• De-duplicate credentials. If you add an API key already in our system, we'll use the existing instead of creating a new one.
• Fixed issues when using foreign keys with a schema other than public.
• Properly remove foreign keys when toggled off.
• Fixed an issue with longer Stripe backfills.

"Lazy foreign keys" alpha

Foreign keys serve two important functions in databases: They provide "referential integrity" or the confidence that if one row references another, that other row exists. And they model the relationships of your database. Tools and clients like ORMs use foreign keys to determine how tables and columns in a database relate to one another.

Adding foreign keys to a table that is synced from another source presents a unique challenge. This is because the ordering of inserts and updates is not guaranteed. When inserting a new row, we can't be certain all the rows it references have already been inserted.

We have a few ideas for how to solve this. We've released our first approach into alpha. We call it "lazy foreign keys." The idea: if we insert row A and row A references row B but row B hasn't been inserted yet, we'll still insert row A but we'll remove the reference to B. When B is finally inserted, we put the reference into place.

It's not a perfect solution; this means there will be race conditions around foreign key columns. But we're looking forward to hear how this approach works for customers so we can iterate from here.

Fixes and improvements

• Fix issue of accounts not fully activating after subscribing.
• De-duplicate API keys and credentials.
• Fix Stripe subscription_item deletes.
• Fix situations where the "Login" and "Sign up" buttons on landing page would swap.
• Added docs around our updated usage-based pricing model.
• Add a "Skip" button during on-boarding.

Events Alpha

Often, you need to write code that responds to creates, updates, or deletes in an API we sync for you. You want to know as soon as a payment fails in Stripe, or when a status changes in Airtable. This typically requires polling the API, setting up Webhooks, or doing both.

Sequin's events let you skip all that work.

Events are a durable event stream right in the Postgres database we sync for you. When you opt in to events, you'll see a new table in your database called _sync_event that logs every change in the API. You can work with these events directly in your database or use one of our client libraries to trigger as soon as data you care about changes.

Please send us a note if you'd like to start using events.

Learn more about events in our docs.

Fixes and improvements

• Improved the onboarding experience to include the proxy, /wait endpoint, and events.
• Improved Airtable syncs to handle schema migrations in place, without dropping tables. This new sync process allows you to add views, triggers, and indexes to your database out of the box.
• Added the ability to request an SSH tunnel into your database.
• Fixed an issue that caused sync statuses to show an error during schema rebuilds.

Automatic name translation with the Airtable proxy

When you are writing through our Airtable proxy, we can now translate table and column names from the Postgres snake_case to the Airtable name the API expects. This means that throughout your code, you'll be able to consistently reference Airtable fields using the name format in your database.

To use this feature, just send along the header sequin-translate: 1.

We hope this makes your code more readable, testable, and easier to write. You can learn more in our docs.

Fixes and improvements

• Improved the user experience when selecting which tables to sync from Stripe by adding grouping.
• Improved Airtable syncs. Some larger Airtable syncs received a 30-40 second speed-up.

Choose which tables to sync

You can now pick and choose which tables from a source sync to your database. Often, you only need a couple tables syncing to your database to start building the features your customers need. So instead of syncing all the tables from a source, you can now select just the tables you need. At any time, you can change which tables you are syncing. In the future, when you need more data from a source, you can simply edit your sync and we'll handle the rest.

Fixes and improvements

• Added sub-account support for Twilio. You'll now find that all the data for sub-accounts is syncing to your database in real-time along with a new accounts table.
• Added support for the /alerts endpoint to Twilio.
• Added support for Shopify variant metadata.
• Improved the in-app docs to dynamically show your sync's specific /wait endpoint URL.
• Improved the look, feel, and copy of our landing page.

Fixes and improvements

• Added support for versioning databases. This means we can make changes or upgrades to how we sync from APIs without affecting existing database schemas.
• Add support for client-side SSL certificates. Learn more in our docs.
• Added documentation for the /wait endpoint.
• Added documentation for working with permissions, non-synced tables, and views.
• Added support for Stripe restricted API keys.
• Improved the self-hosted database provisioning flow by adding Sequin's whitelist IP addresses.
• Improved the API credential experience. You can now select your credentials from a clean list of previously validated keys.
• Fixed an issue that caused non-synced tables in a schema to drop.
• Fixed an issue in the Sequin console so that only synced tables are displayed.
• Fixed an issue that caused all statuses in the console to errantly glow red.

Fixes and improvements

• Published a blog post on how to use PG_NOTIFY to handle events with Twilio syncs.
• Added support for using Airtable table_id and field_id as Postgres table and columns names. Using this setting will make your database less readable directly, but will mean that code connected to your database will be resilient to table/field name changes. Send us a note for early access.
• User experience improvements and bug fixes when connecting a self-hosted database.
• Improved the organization of our docs by listing playbooks alongside their associated source.
• Fixed issues with connecting to Stripe using OAuth.

Fixes and improvements

• Published a tutorial on how to use Twilio syncs with Retool
• Published a blog post about how we dog food our Stripe sync
• Fixed login and sign-up jitters in the header
• Improved the self-checkout page
• Improved our landing page copy and images
• Improved the /wait endpoint by dropping the kind parameter
• Introduced graduated pricing for Airtable syncs

Sync Twilio

Twilio is now ready for syncing! We'll sync messages, phone calls, and recordings right to your production database. Integrate faster by using SQL or your favorite ORM to work with all your conversation data. Learn more in our docs.

User management

You can build faster when everyone on your dev team has a Sequin account. Team members can set up new syncs (eg., developer databases), edit existing syncs, or pull up credentials for their database or /wait endpoint. We've supported teams since we launched, you just needed to ask us to merge your team together into the same organization. Now, you can do this all on your own! Just click on the "Users" tab and you can invite and manage the users on your team.

Fixes and improvements

• Added support for Stripe OAuth
• Added self-checkout
• Added ORMs and the /wait endpoint to the connect modal
• Added ORMs, self-hosted databases, and the /wait endpoint to our docs
• Added platform specific sources pages to the website
• Improved the sign-up survey
• Improved the experience for setting up self-hosted databases

The /wait Endpoint

With the /wait endpoint, we're completing the write story for new sources like Stripe, Close, and Shopify. (Airtable already supports writes via our special Airtable proxy.)

We advocate for a one-way data flow: data flows from the upstream platform to your Sequin database. You read from your database, but write changes back to the upstream API. Those write changes you made will flow down to your Sequin database for you to read again.

Sometimes, you want your code to read from your database right after making a change via the API. But there's a race condition: it can take a second or two for that change to propagate to your database.

So, to ensure a recent write – like a new Shopify order – has been synced to your database, you'll call our /wait endpoint. /wait returns a 200 ok only after we've confirmed that your database is up-to-date and in-sync with Shopify.

Close is generally available

Close is now generally available as a new source! Sync all your leads, contacts, and opportunities right to your production database.

Fixes and improvements

• Added support for Airtable thumbnails.
• Added a real-time backfill progress indicator for new syncs.
• Added a trial duration banner to the console.
• Added automated trial expiration notifications.
• Added an option to halt syncs when an associated view has a conflict.
• Improved the sources section of our website.
• Improved navigation in our docs.
• Switched to flat rate pricing with 14-day trial period.

Shopify and Close integration

Our support for Shopify and Close is in closed alpha! If you'd like early access, send us a note. They'll be public soon.

With our Shopify sync, all your customers, products, and orders replicate to Postgres in real-time. Build new front-ends, internal tools, and more on your Shopify data using SQL or your favorite ORM.

Close is a popular CRM (we use it here at Sequin!) Just enter your Close API key and we'll begin syncing your contacts, leads, and opportunities right to your database. Having your CRM data in your database makes it easier to automate your pipeline, analyze sales, and build internal applications for sales and marketing.

Fixes and improvements

• Added foreign key support for Stripe.
• Added a sources section to our website.
• Improved voting for next Sequin sources.
• Renamed "Optimize sync" to "Turbo sync" for Airtable – it's more accurate.
• Improved onboarding by adding helpful emails.
• Improved the Stripe schema used by Prisma. We shortened/cleaned-up names for backrefs and forwardrefs.
• Added this changelog to our website :)