Add clamd config option to force blocking clamd database reload to
conserve RAM. Users may set `ConcurrentDatabaseReload no` in their
clamd.conf config file to force a blocking reload.
The blocking mode will still perform the reload in a new thread, but
will first free the current database, wait for scans targeting that
database to complete, and then load the new database in the new thread
and wait (`pthread_join()`) on that thread. Once loaded, any pending
scans will continue. This is effectively the same behavior as how
clamd reloads worked before the multi-threaded database reload feature
was added.
This makes clamd responsive to simple (version,contscan,etc.) requests
even during multiscan.
Previously these would get stuck behind a ~100 item queue, and the 1:4 ratio
of executing these commands wasn't working, since the commands weren't in the queue
in the first place.
Each multiscan command needs 1 control thread, and 1 (possibly shared) scanner
thread.
If the number of multiscan control threads would be equal to MaxThreads no
progress could be made, since each one would be waiting for 1 free scanner
thread (but none can be spawned since maxthreads was already reached).
As long as there is 1 thread free for scanning, we can have N-1 multiscan
commands active, with N MaxThreads.
since a buggy client implementation may get stuck in send(), and then clamd
gets stuck in send() -> deadlock. To avoid this we need nonblocking sockets, and
(low) timeouts on send.
Add more tests for clamd protocol, including a stress test for IDSESSION.
git-svn: trunk@4798
handle multiple %v parameters
introduce poll_fds that also does the read into a buffer
introduce thrmgr_group* commands for keeping track of groups of commands
(multiscan, IDSESSION)
introduce 2 queues in thrmgr, multiscan queues get lower priority. Commands
are processed from both queues in a 4:1 ratio to avoid starvation.
unify reply code: conn_reply*
unify scanner code into scan_callback that is called from cli_ftw
multiscan doesn't need stat() now if readdir() provides dt_type
redesign clamd main loop: there is now an accept thread, and a
recv()/dispatch thread, +MaxThreads worker (scanner) threads.
introduce limiting on commands when worker threads are contended.
introduce IDSESSION, a replacement for the old SESSION command, that allows
asyncronous (scan) commands and scan replies
introduce INSTREAM that allows sending the data to scan on same connection
(vs STREAM that requires another port)
introduce zCOMMANDS
git-svn: trunk@4755
Also introduce MaxQueue, this will prevent clamdscan from flooding
clamd with too many FILDES requests.
git-svn-id: file:///var/lib/svn/clamav-devel/branches/clamd-proto@4710 77e5149b-7576-45b1-b177-96237e5ba77b
new top-like tool using ncurses: clamdtop
it shows what clamd is currently scanning (and since how long),
including which file(s) multiscan is working on currently,
how many items it has in the queue (and since how long they are waiting),
how many active/idle threads it has,
memory usage statistics,
can monitor local or remote clamd
git-svn: trunk@4324
enable entconv by default
others.c: fix compiler warning
regex_list.c: remove unused code, because a better solution is scheduled for 0.94 (bb #725)
git-svn: trunk@3627
Micah Snyder (micasnyd)