OnFinality Builders Program Application - Robust, Reliable Public Infrastructure for the Shiden Network

TLDR; OnFinality is a leading IaaS (infrastructure as a service) platform providing mission critical infrastructure to the Polkadot ecosystem. We are proposing to run geographically distributed public Shiden RPC API Service on our platform, allowing free community access.

This service is currently online and can be connected to via these endpoints;

  • https://shiden.api.onfinality.io/public
  • wss://shiden.api.onfinality.io/public-ws

About OnFinality

OnFinality is a SaaS platform providing infrastructure services for some of the largest blockchain and cryptocurrency projects in the world, including Polkadot, Kusama, Acala, and Compound Finance. Our mission is to support all blockchain organisations in the world by providing core infrastructure so they can focus on their core businesses.

  • Our API Service can be completely free (within fair usage limits)
  • We’ve provided mission-critical (scalable, and reliable) services to Polkadot, Kusama, Karura, Compound Gateway, and other Substrate based networks.
  • We are the largest IaaS project in the Polkadot/substrate community, serving over 150 million JSON RPC requests to our public API service every day.
  • We provide a one-click-deploy node service that allows customers to deploy their own Shiden validator and collator nodes in our platform.

Motivation

Setting up blockchain infrastructure is difficult, time consuming, and expensive:

  • It requires a level of server development expertise that many do not have. A Shiden node for example, requires knowledge of SSL certificate handling, websocket management, firewall configuration, and more to setup, run, and monitor a production node.
  • It’s extremely costly to run a full Shiden node nearly continuously, especially when your traffic is low and inconsistent.
  • Running production level infrastructure is especially tricky. You need to autoscale quickly to handle bursty traffic, and you want to provide services in different regions around the world to provide low latency services - all of this is incredibly costly, but comes with significant economies of scale for a provider.
  • DevOps requires constant attention - time that would be better spent elsewhere building something.

“We’ve worked with other infrastructure providers but none come close to the reliability and performance of OnFinality - their team is an excellent and responsive partner to Acala. Ultimately OnFinality’s community API service allows me to sleep at night.” - Bryan Chen (Co-founder at Acala)

As demonstrated by us in the Polkadot domain and Infura/Alchemy in the Ethereum domain, a shared node API service helps mitigate these barriers of entry to trying out a new protocol or network by reducing all these costs to near zero. We manage all the nodes for our users and distribute them across the world (with intelligent routing) to achieve global scalability from day 1, we monitor each node and ensure that they meet certain service levels, and we have the expertise and scale to handle production workloads and high peak traffic.

The difference between what we are trying to achieve and what Astar already offers via it’s public Shiden nodes is that we are building these to scale for large production applications.

In summary, we try to reduce the pain of getting started on a new network and minimise the costs of accessing secure and reliable nodes while for Shiden’s community.

Proposal

Scope

Firstly, we see this as a long term partnership between Astar and OnFinality. Over time, we aim to return insights and learnings from our experience running this infrastructure to Astar. Our goal is to become a trusted partner and a center of excellence for Astar.

OnFinality can configure and run multiple clusters of Shiden nodes for your community to access under fair use limits.

  • This funding grant consists of two key components:
    • Ongoing running costs of these pools of Shiden nodes
    • Constant monitoring, scaling, optimisation, reviews of the infrastructure
  • Currently we only have a single cluster in Tokyo. To achieve global distribution we recommend running at minimum of 2 clusters (choosing between North Virginia, Tokyo and Ireland), each with a pool of 3 Shiden nodes (public network with a full archive).
  • We apply fair use limitations to our service to mitigate against DDOS attacks. This includes a combination of daily request and rate limits that can be removed for certain partners on request.
  • We’ll maintain the one-click-deploy dedicated node access to Shiden Network for customers that want to deploy their own dedicated validator or collator nodes

Outcomes

There are three main outcomes we believe our service will provide the Shiden Community:

  • Help new users enter the Shiden/Astar ecosystem by providing a free API service and one-click-deploy validator or collator nodes.
  • Through a partnership with OnFinality, provide much needed redundancy for this important community service
  • Improve the resiliency of the Shiden network by reducing the load on the current public Astar endpoints and therefore the effort that the core Astar development team is spending on managing this service (so they can focus on bigger things)
  • Help scale existing production applications in the Shiden ecosystem by providing scalable infrastructure services that can be relied on
  • Assist with the testing of the Shiden network by providing valuable data back to the community on how nodes are running and scaling as the network traffic increases, and what kind of requests are commonly made

Cost Breakdown

Ongoing Quarterly Costs

It’s important to note that the ongoing running costs are a current estimate. As part of our service we actively manage the clusters of Shiden nodes, scaling and managing them in real time to provide consistent and reliable service. OnFinality will be in regular contact to discuss usage and cost optimisations that OnFinality are carrying out.

The following table shows the forecasted price for a single cluster of Shiden nodes (all prices are in USD). The ongoing costs will be submitted at the end of each quarter and are based on the following prices.

Type Unit String Unit Price Monthly Hours Quantity Monthly Estimates (per Cluster) Quarterly Estimates (per Cluster)
Node Costs $0.15 / unit / hr $0.15 730 8+4+4=16 $1,752.00 $5,256.00
Storage Costs $0.0005 / GB / hr $0.0005 730 60*3=180 $65.70 $197.10
Backup Management Fee $30 / month $30 1 $30.00 $90.00
Backup Storage Costs $0.0005 / GB / hr $0.0005 730 60 $21.90 $65.70
Data Traffic $0.15 / GB $0.15 ? ? ?
$1,869.60 $5,608.80

Total expected ongoing quarterly costs per cluster: ~$7,974.00

3 Likes

Hi, @jamesbayly James!

Sounds like a highly productive proposal👍 Those participants, especially Astar Official Ambassadors should all participate in this and run our nodes and collators🤝

Just wonder if you guys can provide any demonstrations on the on-click-deploy feature via zoom call so that we can have some AMA sessions as well. Other than that, no other comments.

We can absolutely provide demonstrations, what timezone works best for your community?

Thanks, James. It would be wonderful if you could take a record on Youtube and make it public on this forum so that everyone can review and understand how good OnFinality is!

4 Likes

Thank you for this detailed proposal, onFinality provides a great service and I would love to see a long term partnership.
However I am not sure I get the numbers well: how can quarterly estimates be lower than monthly estimates for a single cluster? Did I miss something there?

1 Like

@bLd759 You found a mistake in my calculations - not sure how I missed this!

1 Like

Revised Ongoing Quarterly Costs

It’s important to note that the ongoing running costs are a current estimate. As part of our service we actively manage the clusters of Shiden nodes, scaling and managing them in real time to provide consistent and reliable service. OnFinality will be in regular contact to discuss usage and cost optimisations that OnFinality are carrying out.

The following table shows the forecasted price for a single cluster of Shiden nodes (all prices are in USD). The ongoing costs will be submitted at the end of each quarter and are based on the following prices.

Type Unit String Unit Price Monthly Hours Quantity Monthly Estimates (per Cluster) Quarterly Estimates (per Cluster)
Node Costs $0.15 / unit / hr $0.15 730 8+4+4=16 $1,752.00 $5,256.00
Storage Costs $0.0005 / GB / hr $0.0005 730 60*3=180 $65.70 $197.10
Backup Management Fee $30 / month $30 1 $30.00 $90.00
Backup Storage Costs $0.0005 / GB / hr $0.0005 730 60 $21.90 $65.70
Data Traffic $0.15 / GB $0.15 ? ? ?
$1,869.60 $5,608.80

Total expected ongoing quarterly costs per cluster: ~$5,608.80

@sota can you please update the post with the corrected numbers?

3 Likes

Thank you for the update @jamesbayly and comments @bLd759

I have updated the number :smiley:

Great Idea! That will help a lot @jamesbayly
Thank you in advance.

Thanks for the update, it looks good right now :slight_smile:

Based on the work flow, let’s have a poll this week!

Name: OnFinality Builders Program Application
Right to Vote: Core team and ambassadors
Opening Time: August 5th
Ending Time: End of August 8th

OnFinality Builders Program
  • Yes
  • No

0 voters