Firstly, this is a really tough situation to be in. I'm sorry you're going through this.

He thinks I'm being too restrictive by need to put a label on our relationship, but I feel very insecure when we just "go with whatever"

You're allowed to have needs. Especially since he's also asking for significant changes to your situation in support of his own needs. If using some kind of label helps you feel more secure within your own understanding of the relationship, then I would say that him tolerating your usage of those labels is a reasonable compromise.

How can I handle living with him and shifting my feelings from romantic to friendship?

I wish I could offer more hopeful advice for you, but truly, authentically shifting your feelings about someone isn't always possible, and it's sadly never easy. This doesn't mean that it's impossible to do, or that it will be impossible for you to do so, and/or find another solution that allows you to live with those feelings.

For me, I am still in love with someone who doesn't fully reciprocate those feelings because we have very different personal goals for the kinds of relationships we want in life. I've had to go through varying stages of grief, relapses of that grief, and other kinds of nonlinear processing about my feelings, the dreams I once had for what our life could look like, etcetera. I learned the hard way that attempting to "force" myself to stop feeling romantic feelings for him does not work, and I've given up on believing that I'm actually going to fall out of love with him.

What's changed for me and made the biggest positive difference is that I've learned to believe that I'll be okay regardless. I've also previously tried going no contact with this person as an experiment, and learned from that I would rather live with him in my life, knowing that our feelings for each other will be as they are going to be and we can still enjoy each others presence in our lives, rather than have him gone forever (because quite frankly, that felt like I was almost mourning his death). I've also found some comfort in recognizing that, even though his feelings for me aren't exactly the same as what I feel for him, that he does still love me in a companionate sense and that is made evident through his behavior and continued investment in our friendship.

In contrast, there was someone else I was in love with, and in a long term monog relationship with, who treated me very poorly in that relationship. For a while I believed I had fully fallen out of love with him, but over time I eventually came to realize that there are still lingering feelings there that will never truly fade either. What makes that sensation different though is that I have no desire to reconnect with him or rebuild a relationship.

So what this could look like for you--

1. You may want to assess your own experience with romantic feelings in general, and how your feelings for your live-in partner feel to you. If you come to the conclusion that you don't view yourself as capable of falling out of love with this person, learning to accept the presence of these feelings without guilt is going to be important for helping you move forward.

2. Definitely focus on what kinds of boundaries will help you manage your own feelings. It's kind of like making dietary changes or cutting out alcohol-- if you determine that something you participate in together has some kind of consistently negative/agitating emotional effect, or makes it harder for you to get by with the basic experience of living together, then you may want to consider cutting back on those things.
Do, also, continue seeking support from your other partners. Assess if there's anything that you were going to your live-in partner for that meets specific needs, and try to seek fulfillment of those needs from other relationships.

3. Make a habit out of reassuring yourself that you are making the best choices for you. After leaving the relationship where I was mistreated, I still spent a while having to convince myself that I was making the right choice by staying away. "Trust that letting him go is the right choice / healthiest choice" was the specific mantra that helped me.

I'd also recommend considering having the two of you see an ENM-specialized couples therapist, for a few sessions, if you can track one down. They might be able to help the two of you come up with adjustments that work for both of you.

And hang in there. This is going to feel like plucking eyebrow hairs, but I promise that it will eventually feel easier to manage.