Postmortem for AAVE V2 AMPL liquidity shortfall

This is a postmortem for the AAVE V2 AMPL liquidity and balances discrepancy that led to the inability to withdraw AMPL deposits for some users. To serve as context reference for further discussions on the topic for the Ampleforth DAO.

Note: The aAMPL token is terminally frozen and distributions to cover user balances were made by the AAVE DAO in two proposals:

Proposal 1
Proposal 2
Overall, 60% was funded by AAVE DAO and 40% was funded by the Ampleforth Foundation

Timeline of Events:

  • The aAMPL custom implementation was developed by the Fragments team (Ampleforth developers) and audited by Peckshield, who have extensive experience with AAVE contracts. The AAVE dev team also reviewed the implementation. That’s before BGDlabs took over AAVE development.

  • July 2021: The market was launched.

  • May 2022: A discrepancy between balances and available liquidity was noted. This issue was discussed with BGDLabs and monitored by Ampleforth. At the time, the discrepancy was minor and well below the AMPL reserve balance.

  • October 2022: It was concluded that the accumulation of small interest errors, typically negligible for other AAVE markets, became significant for the AMPL market due to the pool APY reaching up to 186210.38% at 100% utilization. BGDLabs and Ampleforth agreed that the aAMPL V2 pool would be frozen during the transition to AAVE V3. If the pool became insolvent before all deposits were withdrawn, BGDLabs would propose using the AMPL reserve to inject more funds for withdrawals.

  • November 2022: After this proposal was executed, Ampleforth assumed the issue was resolved from their side, expecting BGDLabs to ensure the market remained frozen, eventually drained, and the reserve balance used if necessary.

  • January 2023: The AMPL reserve intended for contingency was liquidated in this proposal. The reserve was approximately 283,500 AMPL (~47,250 WAMPL).

  • December 2023: Over a year after the market was frozen, BGDLabs contacted us about a discrepancy between the aAMPL total supply and the AMPL liquidity in the pool.

  • January 2024: Our initial proposal to BGDLabs for resolving the larger shortfall was to assume lenders were holding AMPL since a date before the discrepancy grew exponentially. This aimed for a simple, non-controversial resolution, avoiding off-chain simulations of AAVE platform behavior. BGDLabs cited potential overpayment issues, suggesting a full behavior simulation was necessary to determine exact balances. Because the exact source of the issue had still not been identified, that task would involve simulating AAVE’s lending/borrowing logic.

  • February 2024: We could not isolate any issue in the aAMPL implementation. We reiterated to BGDLabs that the most effective resolution was using AMPL distributions based on the full rebase amount from the pool freezing date. At that point, the total liability was around 49,412 WAMPL at ~$8.5, which could have been mostly covered by the liquidated AMPL reserves and, if necessary, the Ampleforth Treasury. BGDLabs did not support this proposal, advising instead that Fragments simulate balances based on specific holder transactions.

  • March 2024: Despite significant time investment, we could not pinpoint the discrepancy’s root cause. However, we determined that adjusting the liquidity index with the same ratio for all balances could accurately reflect balances at a particular time, addressing BGDLabs’ overpayment concerns.

Based on that, we posted this on March 9 AMPL problem on Aave v2 Ethereum - #45

In response BGDLab informed us that this proposal is not acceptable as it is proposing distributions in AMPL and assuming full rebases, and that they will make another proposal themselves. At this point it they and ChaosLabs started doing the simulation using their approach.

Our perspective:

As aAMPL and pool reserves are owned and governed by the AAVE DAO, Actions from our side were not possible without support from key voices and voters in the AAVE community.

Even though the issue originated in the aAMPL implementation, BGDlabs missed two opportunities to make this issue inconsequential. Firstly, by allowing the liquidation of the AMPL reserve intended to cover pool insolvency. This knowledge was apparently lost internally during personnel changes. Secondly, by insisting on simulated balances distributed in stablecoin rather than AMPL and using simulated interests to determine balances instead of full rebases. This led to significant delays, higher costs, and frustration for users. The AAVE DAO ultimately faced much higher costs, with the Ampleforth Treasury covering most of the difference.

All these issues arose from an overly rigid stance from BGLabs on distribution calculations and offloading it on Fragments to do those detailed simulations of the Aave platform. And insisting on using stable coins for distribution rather than the asset of the pool.

What we could have done differently:

  • In November 2022, we did not advise depositors to withdraw from AAVE based on assurances from BGDLabs. With the AMPL reserve levels as they were, we had no concerns about their ability to handle the situation. In hindsight, advising users to withdraw would have clarified the situation and avoided confusion.

The original thread on AAVE’s Forum: AMPL problem on Aave v2 Ethereum