Unfortunately, a lot of people get into trouble with Crohn's because the lack of obvious, life-affecting symptoms means that they do not take any meds to reduce inflammation.
Long-term inflammation, even if it is only mild, can lead to complications down the line in future years, potentially causing strictures, obstruction, abscesses, fistulas or even cancer.
You often read about people going drug-free for years and feeling great for it, only to end up on the operating table when the results of the internal inflammation arise. There will also be others who live drug-free successfully without any problems.
I think you need to do as much research as possible so that you can make an informed choice about which is the best option for you. You may prefer to carry on as you are if you feel well, but only if you are aware of the risks of that decision, and consider the benefits to outweigh any side-effects of drugs.
I do sympathise as I am in a similar situation. Currently my symptoms are very mild, but they are still not fully clearing on the meds I have tried so far, and I am now looking at having to start Humira soon. I'm trying to balance the long-term risks/benefits of such a strong drug against those risks/benefits of low-grade inflammation. I feel the decision would be much easier if my symptoms were worse.