Astar Network Post-Mortem Report: Critical Vulnerability Response (May 2025)

Overview

This report is being published to provide transparency and clarity regarding a critical security vulnerability that was identified in the Polkadot SDK and affected multiple parachains in the ecosystem. On Friday, May 23, 2025, the Astar team was informed of the vulnerability, which had already been exploited on another parachain (Bittensor), and posed a direct threat to the operational integrity of Astar Network by allowing transactions that could halt the entire parachain.

Upon being alerted, our team swiftly investigated the issue, verified its impact, and developed a secure patch. Runtime 1501 was deployed across Astar, Shiden, and Shibuya to eliminate the vulnerability. This patch retained all the original features of Runtime 1500 and introduced a critical fix.

:warning: Importantly, no exploit occurred on Astar, and no user funds or operations were compromised throughout this period. :warning:

The rapid response and immediate deployment of the fix ensured that the network remained secure and fully operational at all times.

This post-mortem provides a detailed explanation of the vulnerability, our rapid response including the decision to deploy Runtime 1501 via a sudo call, and the rationale for each action taken. Our goal is to transparently share this information with the community, uphold accountability, and reaffirm our commitment to security, resilience, and decentralized governance.


Summary of the Vulnerability

A severe vulnerability in the Polkadot SDK allowed the construction of a transaction using a nested call. If executed, this transaction could generate a faulty block, and as this faulty block propagated across the network, it would stall or completely halt a parachain. This issue had already caused a halt in the Bittensor parachain, raising the alarm across the ecosystem.

The same vulnerability was present in Astar and, if left unpatched, could have been exploited to similar devastating effect. The severity of this risk was confirmed shortly after the initial discovery when we received a bug bounty submission through Immunefi. The report included a working proof of concept developed by white hat hackers, validating that the vulnerability was exploitable and had already been independently identified. This significantly elevated the urgency of our response, as the presence of a public POC suggested that bad actors could also become aware of the exploit at any time.

References:


Timeline of Events

Friday, May 23 (UTC):

  • 3:20 PM – Initial report of the vulnerability received from the ecosystem
  • 3:40 PM – Immunefi Report received, confirming the issue and including a working POC by White Hackers
  • 7:15 PM – Patch (Runtime 1501) successfully built & tested
  • 7:58 PM – Sudo call initiated on Shiden
  • 8:04 PM – Shibuya upgrade enacted → Subscan Link
  • 8:17 PM – Sudo call initiated on Astar
  • 8:28 PM – Shiden upgrade enacted → Subscan Link
  • 8:37 PM – Astar upgrade enacted → Subscan Link
  • 9:23 PM – Shiden upgrade confirmed successful :white_check_mark:
  • 9:41 PM – Astar upgrade confirmed successful :white_check_mark:

Total Response Time: ~6 hours and 20 minutes from report to full patch deployment.


Governance Context and Emergency Response

Governance Mechanism on Astar

To enact any runtime upgrade on Astar Network, proposals must pass through the on-chain governance process. This process involves submitting a preimage, launching a referendum (either via public or external proposal), completing a voting period, and awaiting enactment after approval. Normally, the full process can take over 9 days (7 days voting + 2 days enactment). However, emergency mechanisms exist:

  • The Main Council can submit external proposals.
  • The Technical Committee can fast-track referenda, significantly shortening the voting and enactment periods.
    • A standard fast track (2/3 Technical Committee approval) reduces the voting period to a minimum of 2 hours.
    • An instant track (unanimous approval) can reduce the delay to just minutes.

Why sudo Was Used

In this case, while it was technically possible to coordinate an instant fast-track using the Technical Committee, the severity and time sensitivity of the situation required immediate action. A direct sudo call was used to enact Runtime 1501 and mitigate the threat without delay.

Factors Contributing to This Decision

  • Public Knowledge: The Immunefi report confirming the issue included a working proof of concept. This indicated that whitehat hackers—and potentially malicious actors—were aware of the vulnerability.
  • Governance Execution Delay: Even under optimal conditions, the fastest governance path (instant fast-track) would have still required up to 5–7 additional hours including setup, voting, and enactment time.
  • Risk to the Ecosystem: If Astar had been attacked and the network halted, it could have caused significant disruption across the Polkadot ecosystem, undermined user trust, damaged Astar’s brand reputation, and potentially impacted the value of the ASTR token.
  • Referendum #22 was still active: It contained a runtime with the vulnerable logic. If enacted before the fix, it could have placed the network at immediate risk.
  • Coordination Constraints: The incident occurred late Friday evening, when some key governance actors were unavailable or unable to fully participate in an emergency coordination.
  • Weekend Risk: The team prioritized securing the network heading into the weekend, when monitoring and coordination resources would be reduced, and the probability of public exposure and exploitation could increase.

Why Was This Bug Rated as Critical?

The above factors lead the team to designate this bug as critical because:

Easy to Execute

  • A fully functional proof-of-concept was already public in the Immunefi report, so any attacker—white-hat or malicious—could replicate the exploit in quickly.
  • No exotic prerequisites or complex setup were required, meaning even moderately skilled actors could weaponize it immediately.

Massive Potential Impact

  • Exploiting this flaw would allow an attacker to halt the Astar network or corrupt its runtime logic.
  • The financial damage could be significant, user funds could become locked or stolen, and community confidence would collapse overnight.

High Likelihood of Attack

  • With Referendum #22 still in play—carrying the same vulnerable code—any governance enactment could unknowingly “arm” the network with this flaw.
  • Late-Friday timing meant most governance participants were offline, and weekend staffing would be slim, giving attackers a wide window of opportunity before detection.
  • Public proof-of-concept plus reduced monitoring over the weekend made exploitation not just possible but probable.

Taken together—low barrier to entry, catastrophic downstream effects, and a narrow governance patch window under strained conditions—the incident team concluded that only an immediate, out-of-band fix could prevent a near-certain, high-impact attack.

We acknowledge that under ideal circumstances, the governance system could have handled this upgrade with some delays. However, given the urgency and potential consequences of inaction, we determined that immediate mitigation through sudo was the most responsible course. We accept that this decision deviated from our decentralized principles and are taking steps to reinforce emergency governance preparedness moving forward.


:pen: Communication Strategy

We made a deliberate decision not to communicate publicly during the weekend to avoid drawing further attention to the vulnerability.

This allowed time for:

  • Other parachains to implement their own fixes quietly
  • Reduced risk of opportunistic exploits from bad actors by limiting public visibility

We are now sharing this post-mortem to maintain transparency with our community and ecosystem partners.


Current Network Status

  • Runtime 1501 is live across Astar, Shiden, and Shibuya
  • Includes all features from Runtime 1500, such as the dApp Staking threshold update
  • Vulnerability has been fully patched
  • Network remains stable and secure

Context Behind the Cancellation of Referendum #22

Following the identification and patching of the critical Polkadot SDK vulnerability, the Main Council took the decision to cancel Referendum #22 via Council Motion #23.

Referendum #22 had been created to approve Runtime 1500, which at the time of submission did not contain any known vulnerabilities. However, once the SDK vulnerability became known, it was confirmed that Runtime 1500 included the affected logic and was therefore no longer safe to deploy.

If Referendum #22 had passed and been enacted, it would have reintroduced the security flaw that Runtime 1501 was specifically designed to fix. To protect the network from regression into a vulnerable state, the Council acted swiftly to cancel the referendum.

Runtime 1501 includes all of the original features planned in Runtime 1500—such as the dApp Staking threshold update—alongside the necessary patch to address the vulnerability. This ensured both continuity of planned upgrades and the security of the network.

:link: References:


:pushpin: Next Steps

To prevent similar situations in the future and improve our readiness for emergency runtime upgrades, we will take the following steps, with the ultimate goal of preventing the use of sudo and maintaining a fully decentralized governance model—even under time-critical conditions:

  • Define a formal incident management procedure for Astar Emergency Upgrades, involving both the Technical Committee and Governance, with clear step-by-step actions and parameters.
  • Ensure that the Technical Committee and Main Council members are reachable at all times for urgent coordination.
  • Explore reducing the minimum fast-track voting period (currently 2 hours with 2/3 TC signers) to 1 hour to enhance responsiveness during emergency scenarios.
  • Provide simulation training and documentation for all governance actors (Technical Committee, Main Council) to ensure clarity and confidence in executing fast-track governance workflows.
  • Increase transparency around emergency decision-making by pre-committing to public reporting procedures, including publishing post-mortems within a set timeframe after any emergency intervention.
  • Review and streamline technical tooling and documentation related to fast-tracking referenda to minimize friction during coordination and execution.

We appreciate the support of our community and partners during this event. Please direct any questions to our forum or Discord for further discussion.

Astar Foundation & Main Council

9 Likes

Thanks for the quick fix and report.

I agree with keeping sudo for emergency response, but do you plan to destroy it in the future?
I am not a dicentralization maxi, but sudo diminishes the fundamental significance of on-chain governance. I don’t think we need to decide on a specific timing, but I think sudo should be scrapped once we have a system in place and know that emergency response will not be a problem in the future.

Hi @you425,

Yes, removing the Sudo pallet from Astar Network is definitely part of our plan. We’ve kept it in place for now to ensure we can handle critical situations effectively, and also to allow time for governance participation to mature and reach a level where the network and community can sustainably manage all scenarios on its own.

While we don’t have a specific timeline yet for deprecating the Sudo pallet, it’s certainly on the roadmap.


Gaius_sama

1 Like

Hello @Gaius_sama san

After reading your post, I realized how serious the situation is.

I thought the only problem was the parachain shutdown, but it seems that user funds could have been locked or stolen.
This is a shocking vulnerability.

I highly appreciate the decision to use sudo this time.
I understand that it was a necessary response.
And I respect the skill of all of you who dealt with the vulnerability in just 6 hours and 20 minutes.

Thanks to the core team.

I am a strong supporter of decentralization, but this incident may change my mind a little.

I understand that you carefully considered how to handle governance in an emergency.
However, I think the flow is complicated and could be confusing.
That’s why I understand that training is necessary.

Another risk is if, for some reason, key person cannot be contacted in an emergency.

Therefore sudo may be a feature that should be left.
Either way, I don’t think there is time for dapp stakers to vote.

I think protecting Astar Network is the most important thing.

3 Likes

Thank you!
I know it won’t be an easy path, but let’s work hard together to remove Sudo.

1 Like

Thank you for the information. I learned a lot of things, both in terms of the causes and the solutions. =)