Consigliere Solves All Our Problems

We use Consigliere to remove performance bottlenecks relating to how we talk to the blockchain

10 weeks ago the DXS growth engine roared to life. Volume on DXS went 20x overnight. Shortly thereafter the platform shat the bed.

What happened?!

We bumped into bottlenecks, all of which relate to how we talk to the blockchain.

We have resolved all of these bottlenecks because we want to win. We’ll tell you how we did this because we want you to win too.

We use Consigliere

Consigliere is primarily an incredibly powerful BSV indexer (secondarily it is a transaction building SDK). Importantly, Consigliere can index, build and interpret STAS transactions.

We need Consigliere for 3 reasons:

  1. We need more performance. 3rd party blockchain indexing APIs have proven to be too slow and unreliable. We cannot afford to be held back when seeking 100x, 1,000x growth

  2. We need more independence. We want the option to run an open source blockchain indexing solution in-house (to eliminate downtime and risk of vendor lock-in)

  3. We need STAS. Tooling needs to support STAS to power Fiorin wallet and USD trading on DXS

What were the bottlenecks?

You should definitely read β€˜How DXS Talks to the Blockchain’. From that, you’ll understand that DXS needs to perform three key functions (extremely frequently). These three functions are checking the mempool for a TXID, retrieving UTXOs for particular addresses and broadcasting transactions.

Checking the mempool for a TXID

If a TXID exists in a miner’s mempool we have high confidence that:

  1. TX will eventually be mined into the longest chain

  2. TXID is immediately retrievable on an independent service such as Whatsonchain (a user can verify their trade details independent of DXS)

We run this check every time DXS receives a transaction from an integrated Web3 Wallet (again, reading this post will serve you well).

β€œBro, just implement P2P TX and Transaction Ancestors (SPV Envelope) as outlined by Dr Craig S Wright and documented by the BSV Technical Standards Committee then you won’t need to run this check, bro”

Bro, I literally proposed the standard that DXS and every wallet on BSV are not using! The practicalities of doing business on BSV have enforced different ways-of-working.

How did DXS check the mempool before Consigliere?

We have used Mattercloud, Blockchair and Whatsonchain (WOC) APIs at different points in the past. Most recently we have been relying completely on WOC.

Key issues:

  1. Due to being inconsistent with the blockchain (or down), WOC gave us issues we had to manually resolve every second day on average

  2. WOC average response time was too slow (even 2 seconds is a deal-breaker for us). We typically bumped into this issue at the end of trading sessions when we settle profits for 100s / 1,000s of profitable trades

  3. WOC do some request throttling (by default they allow 3 requests / second) which caused problems for us upon the release of Bounty Trading

Solution

Use Consigliere. 100ms response times, massively concurrent, zero downtime (so far), inexpensive.

Retrieving UTXOs for a particular address

DXS needs to do this every time we make a payment in order to build transactions that:

How did DXS retrieve UTXOs for a particular address before Consigliere?

We have used Mattercloud, Blockchair and Whatsonchain (WOC) APIs at different points in the past. Most recently we have been relying completely on WOC.

We experienced the same three issues as with checking the mempool (above). These issues were exacerbated by the fact that when we ask an API endpoint for UTXOs, these APIs don’t support filtering or pagination. They will simply return all UTXOs for an address (unnecessarily large response).

β€œBro, just keep an internal database of all UTXOs for the addresses you care about, you won’t double spend yourself, bro”

That might seem intuitive on the face of it but it’s much better to generalize this problem down to the root. Once you do this, you can reduce your interaction with the blockchain down to a few standardized API endpoints provided by an independent service. Very valuable if the underlying network (BSV) scales to the point that only miners can afford to run such a service, bro.

Solution

Use Consigliere. 100ms response times, supports filtering / pagination, massively concurrent, zero downtime (so far), inexpensive.

Broadcasting transactions

We have used Mattercloud, Blockchair and Whatsonchain (WOC) APIs at different points in the past. Most recently we have been relying completely on WOC.

Once again, similar issues as above. Yes, I know we’re getting repetitive, stick with me.

β€œBro, just use MAPI, bro”

I would suggest a consultation with Ivan Mlinaric (HandCash) before going full MAPI, bro:

Solution

Use Consigliere. Consigliere is connected to a pruned BSV node. Just put the transaction straight into the mempool. Job done.

Bonus. You’ve already done your mempool check by default. Your UTXO DB is updated by default.

Bonus. 100ms response times, massively concurrent, zero downtime (so far), inexpensive.

I want to use Consigliere for my application!

Excellent. If you want to run Consigliere in-house, the code will be open source very soon. If you’d prefer to outsource your blockchain interactions, get in touch with this guy to make it happen.

I’m Taal or GorillaPool and I want to do better

Run Consigliere and provide it as a service to your customers! Consigliere would be expensive for us to run in-house if BSV throughput grows 100x, 1,000x.

I’m Tokenized

You can easily extend Consigliere to index Tokenized transactions.

I’m HandCash and I don’t run a node

You deserve more reliable 3rd party blockchain infrastructure. Encourage your providers to run Consigliere.

Last updated