I was on a 16" MacBook Pro M1, 16GB, 1TB, and had the feeling that I was often running up against the memory limit. Despite the fast SSD you still feel it; the sharpness of the system is somewhat diminished.
My type of use is the entire MS Office suite (Excel, Powerpoint, Teams, Outlook) all day long, heaps of Safari windows, and regularly starting up a virtual environment via parallels (windows or linux). Everything continued to run relatively smoothly but you still felt that it was work.
With this Mac mini M4 I opted for the 24GB version and what a difference that makes. Of course the processor is a lot faster but you pay less than half of a MacBook Pro (I keep the M1 on when I'm on the road and working with customers), and I can clearly see that I always have 3-4GB of memory space left (in the past it was always used up and a big chunk of swap on top).
I would have preferred the 1TB version, but that was nowhere to be found in the following weeks, so I just opted for 512GB plus an external Samsung SSD of 2TB.