Well, it's doing plenty of I/O. With a heavy load on the machine and not much priority for you, it can then take time. Don't see that ADRDSSU need be much different. If it is accessing catalogs, updating catalogs, it'd be the same - except, perhaps, ADRDSSU is a much more modern piece of software than IDCAMS, so maybe there'd be some difference.
I guess you'll have to experiment. Try removing say 10 files with IDCAMS. Then knowing how many EXCPs that took, try to get to about 100 EXCPs. More if you like. Different types of files, if that is what you are doing (QSAM, VSAM, whatever).
Once you have a nice little trial set, then try with ADRDSSU and see if there is a difference.
Remember to copy your selections first.
The IEFBR14 can't do "generic" deletes. So, even if generated DDs, someone could always slip a new file in. I suspect it would be "faster" because the I/Os would be related to the operating system rather than "your program". Perhaps you can continue the experiment with IEFBR14 just for knowledge.