What is a blockchain oracle?What does it do?
Why is it so important?
An oracle is a method for bringing data from the world outside the blockchain onto it, from ‘meat-space’ as it’s been referred to. This commonly involves facilitating the interaction of Web 2.0 APIs with Smart Contracts to expand the capabilities and possibilities of blockchains.
This sector of the space has been developing for a few years now, and notable examples include Chainlink and Band Protocol, to name but a few.
In many ways, this mirrors an ancient expression of humanity’s search for meaning and purpose from beyond the physical realm.
Am I going off on a wild tangent here? Not really.
API3 seeks, like other projects, to bring data from one realm – the space outside a live blockchain into another realm – the onchain realm (meat-space<—>onchain); it seeks to act as a binding conduit for data between those realms so that data from one realm can interact meaningfully with another.
Allegorically speaking, this mirrors humanity’s constant effort throughout human history to bridge an informational connection with divine, non-physical or metaphysical realms, by means of revelations, prophecies or oracles. This satisfied the urge to derive meaning from another informational realm (the non-physical, non-corporeal), to predict the required actions to achieve a desired outcome, and to gain advantage to achieve desired goals.
A prime example of this would be the Pythian Oracle at Delphi in Ancient Greece who was consulted by the fleet that set out for Troy in Homer’s Iliad – by the Athenians and Spartans before they engaged the Persians – and by Alexander the Great.
We can be grateful that nowadays we have more precise means of bridging the gap between realms of information from which we can gain advantage than were available in those times.
Software is also easier to deal with than chicken entrails, or the flights of birds were in classical Rome, for example.
So then, what can we say of API3, the newest project to build in this infrastructure space, bringing non-blockchain data into the realm of onchain smart contracts?
The first thing would be to define the unique elements that define this project, those being the provision of ‘first-party’ oracles (API3’s Airnode) for the first time in this space, and also a uniquely configured governance structure combined with superbly aligned incentives.
API3 seeks to bring data onchain via APIs to maximise its utility when interacting with smart contracts, but one crucial difference is the fact that unlike other projects in this space, the data providers are the ones that directly control their own feeds and handle data requests from those feeds – via API3’s ‘Airnode’ technology.
Text in bold italics and enclosed in quotes is taken directly from the whitepaper (in the article as a whole) :-https://raw.githubusercontent.com/api3dao/api3-whitepaper/master/api3-whitepaper.pdf
“First-party oracles are integral to the API3 solution. This means each API is served by an oracle that is operated by the entity that owns the API, rather than a third-party.“
In fact, this whitepaper is so well-written, it’s quite difficult to paraphrase and parse from, but I will try.
Having oracles operated by the API data-feed providers themselves is a first in this space, and opens up all kinds of interesting possibilities which weren’t possible before.
In light of this, I’m going to try to avoid direct comparisons with other projects because I don’t think they would be entirely fair to either those projects or to API3.
Security – Aspects of this change when using 1st-party oracles, for example the API providers would be signing the data onchain using their own private keys, as well as the data being private by default; no middleman nodes would be able to ‘see’ or parse the raw data itself. This would obscure and thus de-incentivize its unauthorised resale, which is a potential existent problem with 3rd party oracle systems based on the utilisation of middleman nodes. Additionally, any 3rd-party attack surface is removed.
Redundancy – Since the data providers also control the oracles, they are provided by the original secure source of the data with no mediation and so less nodal redundancy is needed, which reduces costs, network latency and fees. An API provider would be more than sufficiently served by simply placing mirrors in each geographic area for verification. This redundancy can be created within the Airnode itself, and the friction-less nature of the interaction between the oracle feed and the smart contracts mitigates any need to provide any sandbox environment (set-and-forget).
Costs and Revenue – Fewer nodes means lower operational costs, and less network latency means lower gas fees. The design aspects of Airnodes remove a lot of the bottle-necking issues encountered by sequential threading of requests (see below – the multi-wallet aspect). Since no middlemen are involved, revenue is usage-based and accrues directly to the API data-feed provider; this is much fairer since they are the ones providing the managed service and the data itself.
The businesses that provide these data-feeds on Web2 are accustomed to a pricing/revenue model based on being fixed, competitive, recurring and usage-based, which is extremely difficult with the 3rd-party oracle model due to the increased overheads of node-based middlemen and additional redundancy. Since costs are lower, more manageable and incurred by the requester this can now be applied in the API3 service for them (0.10c per request/$100 p.m. sub etc), making it easier to integrate and more attractive as regards integration with existing business models. The synergy of requiring little, if any, retooling in terms of staff and resources with blockchain skills (set-and-forget) reinforces this in terms of potentially very high growth of the platform.
Since the oracles are operated directly by the feed providers, their information is directly visible, something not possible with large numbers of redundant middleman nodes. Data providers’ identities are also visible and verifiable. There is no need for workarounds like off-chain signing, although it is perfectly feasible with the Airnode tech, which reduces overheads and drives ecosystem growth.
Set-and-forget – This server-less oracle service is easy to configure with a little help, and new ones can be implemented daily. The plan to accelerate rollout, though, is to semi-automate and scale the on-boarding of new oracle feeds by building a GUI toolset:-
“Borrowing from the OpenAPI Specification format, Oracle Integration Specifications (OIS) define the operations of an API, the endpoints of an oracle, and how the two map to each other. An Airnode user will be able to serve an API over their oracle simply by providing its OIS to their node. Integrations made in this standardized format will be very easy to collect, version and distribute.”
Additional plans in this area include a ‘node dashboard’ for feed providers to monitor their revenue and nodes etc, and a marketplace listing all available providers, what they provide, fees and endpoints.
This promises to help API3 to scale the ecosystem rapidly with a fast-growing and large base of 1st party oracles to compose dAPIs from.
Usage Model Patterns – These will encompass existing ones as well as future ones which were previously difficult to offer via 3rd-party architecture:-
The use-cases for both patterns, and for future patterns, ae only limited by the imagination and creativity of the people who wish to use them, combined with the quality and the nature of the data being provided.
The 800 Pound Gorilla in the room – sequential threading:
I’ve seen quite a few people, including very skilled and technical ones, who initially couldn’t work out how API3 could handle this problem. The issue is that the EVM handles requests sequentially and puts them in queued threads, and this is one of the reasons why scaling is a serious problem on ETH 1.0.
In fact, if the problem had been tackled from the expected direction of decomposing and recomposing the issue then this project could potentially have been crippled. But the approach used was truly a ‘Eureka!’ moment.
Burak Benligiray, API3’s CTO, outlines and teases it here:https://medium.com/api3/the-gordian-knot-called-the-oracle-problem-e9731c55da13
The key to implementing 1st party oracles though was, in my opinion, not engaging with the sequential threading issue in the first place.
Simply put, what API3 does is to process requests in parallel.
A multi-wallet approach is used, where up to 2^256 wallets are generated within a node and the requester receives their own wallet to deal with the request – so each of this large number of wallets/request-handlers works in parallel.The result? –
NO sequential queuing and NO queue-based bottlenecks, and a superb workaround for scaling.
This multi-HD wallet technology is most commonly used in exchanges to provide large amounts of wallets for each user, many of you have seen it in action when you look at your spot wallets on Binance, Coinbase or Kraken.
It’s not new technology, but this manner of application of it is, in API3, unique. As Burak stated in his medium piece, and I agree, it’s a great way of re-enacting Alexanders’ solving the problem of the Gordian Knot.
Lateral, inspired, out-of-the-box thinking.
In the second, and final, part of this series we will deal with the things that complement and reinforce API3’s unique technology:
Part 2 – All things DAO…Governance and Incentives.See you then, Halo out.