While I believe this model will ultimately be beneficial, the truth is, there are too many bad actors to make this be reviewed and come under the umbrella of a regulatory authority.
The crypto exchanges in many ways are doing this, but they are still centralized in many ways. DEX (Decentralized Exchanges) are coming up – but so far, nothing major.
Some of the issues you might face are things like liquidity, price manipulation (trust me it will happen), money laundering or ill-gain money being channeled through (can cause a lot of problems dealing with Police and trying to explain you simple did an exchange, you had no idea that the US Dollars you were buying were tainted).
However, the biggest challenge would be having such an exchange platform attached to your banking network. How you on-load and off-load money from the exchange is where your challenges would lie. The very minute you start thinking of attaching/connecting this network to the banking network, you would then also come under the regulatory framework that governs other banking and non-banking financial institutions.
It is doable, however, plagued with problems. You need to figure out a way to combat money laundering, terrorist financing, price manipulations, bots, liquidity, and identity.