It depends on what kind of server you are running. On Unix based systems rootkits can be installed into a directory called dot dot space (.. ) which looks like the notation for the parent directory.
I seriously doubt that deleting files and folders will do anything to elay your fears. Rootkits can embed themselves deep onto your machine and sometimes can load into the kernel which means that the only way to get back to full health is to perform a complete wipe and reinstall on the box.
Rootkits are notoriously difficult to get rid of. It might look like you've cleaned it all up, only for the rootkit to lie dormant for a few weeks, only to respawn at a later date.
On most linux distros you can install "chkrootkit" and run this on regular intervals. You can set this up automatically and get it to flag what it finds and email you a report. There's tons of these security scanners available.