Astar zkEVM: Network Upgrade Report

if you mint a lot on 7-9 april, thats gone i think
need to re do

As reported on this topic, transactions executed between April 6th and 8th may have been affected by reorganization, processed in alternative blocks, or not processed at all. Therefore, if you had transactions during this period that resulted in acquiring a significant amount of Yoki Rore, we regret to inform you that you will need to perform them again. We believe that the costs, such as transaction fees, have been reverted, but we apologize for the inconvenience caused by the additional effort required on your part.

1 Like

Hey there, still not possible to process transactions on yoki or bridge from astar zkevm, it shows right balance again, but all transactions faul since the day it was supposed to be online again

I admire the work you are doing; it’s great that an update could be made to identify the errors. :clap:

1 Like

Hello everyone,

I’m Shaun, CTO of Astar Network. Here’s a quick update on the Root Cause Analysis (RCA) on the recent Astar zkEVM incident for full transparency with our community:

The Polygon CDK stack utilizes Exit Trees, a specific type of Merkle Tree, to record information about assets transferred out of the network. When a cross-chain transfer occurs, such as sending ETH from Ethereum L1 to Astar zkEVM, the Exit Tree is updated, and a new exit root of the tree is generated.

Due to a bug in the CDK stack, on April 6th UTC, the exit root was affected by a reorged(reorganized) block in Ethereum. This block returned an L1 info root but did not contain the expected update; instead, the update occurred in the next block. This discrepancy led to a faulty batch on Astar zkEVM, where the batch was incorrectly sequenced with an invalid state.

To address this issue, the Polygon & Astar core dev team decided to upgrade the Astar zkEVM mainnet from Fork ID 7 to Fork ID 9. This upgrade would allow for the necessary resync of the invalid batch. The upgrade and resync operation was applied on April 8th (UTC), resolving the issue but necessitating the reorg.

As a result of the reorg, transactions processed between April 6th and April 8th may be affected. These means impacted transactions were invalidated, essentially as if they never occurred on the Astar zkEVM mainnet. For eg: If a user minted an OMA token and paid X amount of ETH as a transaction fee, and if this transaction was affected by the reorg, it would be as if they had not minted the token without spending any ETH for the transaction fee.

A full technical root cause analysis of the incident will be published by the Polygon Team on the Polygon Forum and I will share it here once it’s ready.

Thank you for your continuous support and patience.

10 Likes

Thank you for the additional report. I am very grateful and respectful for everyone’s significant efforts.

I thought the following points were new information.

The Polygon CDK stack utilizes Exit Trees, a specific type of Merkle Tree, to record information about assets transferred out of the network. When a cross-chain transfer occurs, such as sending ETH from Ethereum L1 to Astar zkEVM, the Exit Tree is updated, and a new exit root of the tree is generated.

Where can I refer to for more details on this operation or architecture? I suppose it is probably in the Polygon CDK documentation, but I would appreciate it if you could give me a pointer.
I’d like to learn to understand correctly.

1 Like

The full technical root cause analysis is available on the Polygon Forum:

7 Likes

Quite valuable the information you provide, Chaun, with this and the link provided by Gaiuss, we will be able to be aware of what happened and inform all the rest of the local communities. good job!

Thanks for the report.
Regarding Issue #3.

Is there an estimate how big the impact was?

Thanks for sharing!
I believe the reason for the loss of most of the transactions after the chain was restarted was because the data was deleted from the pool DB. Why were these transactions deadlocked?

There are bugs and slowness in general on the astar zkevm. I don’t know if this is due to the network or the dapps’ own systems. Isn’t 2-3 weeks a long time to fix these problems? There is no reason not to get tired of trying every day to do something… leave the network and turn to other networks. I am writing as a regular user, i am really tired of trying to be active on the network every day and getting errors. So what do you recommend me to do?

Thanks for sharing the details on the forum of both here and Polygon side as well as the details during the community call.

Thank you for this detailed update on the recent incident and the steps taken to resolve it. The transparency and prompt action by both the Polygon and Astar teams are reassuring.

It’s great to see the commitment to keeping the community informed and maintaining the integrity of the network. Looking forward to the full technical RCA to understand more about how such issues can be prevented in the future. Keep up the excellent work!

HELP!!!
I unstaked vASTR token on omni.ls (on Astar zkEVM network) on April 11, 2024 15:47:30 PM (UTC).
Unfortunately, the Bifrost support team informed me that due to the earlier issue with the Astar zkEVM network, they were unable to process my unstaking request and that my 30k vASTR (unstaked amount approx. 32k ASTR) are lost and I should contact the Astar network team to recover my funds.

I submitted this Google form “https://forms.gle/GLWWBH7xkeLZzrY26” 10 days ago, but have not yet received a response.

Please help to resolve this problem and recover my funds as soon as possible.

Here are some information you may find useful
Transaction Hash:
0x6fe331e42066b24ce3d40f2a53fc36213155a42a0aa41ddf57b6758a2ec0ee5d

Unstake Time:
Apr 11 2024 15:47:30 PM (UTC)

Unstake Amount:
30298.752039vASTR (to approx. 32k ASTR)

Thank you very much for you help.