Prototypes (or the declarations themselves, if there is no
corresponding prototype) for functions that take no arguments are
required by the C standard to specify (void) as their argument list;
for example,
regex_pcre.h:79:1: error: function declaration isn't a prototype
[-Werror=strict-prototypes]
79 | cl_error_t cli_pcre_init_internal();
Future versions of clang may become strict about this, and there's no
harm in conforming to the standard right now, so we fix all such
instances in this commit.
The clamonacc-ddd thread (clamonacc/inotif/inotif.c) has currently no
handling for inotify events where the passed watch descriptor is lower
than zero (e.g.: "-1"). Although not actually mentioned in the inotify
documentation, this case has actually been observed:
[...]
ClamInotif: inotify event wd:-1, mask:1073742080, cookie:0, length:16, child:<CHILD-PATH>
Clamonacc: onas_clamonacc_exit(), signal 11
Assumption: This probably occurs when a inotify event is generated for
the creation of a directory including subdirectories. The clamonacc-ddd
thread then discovers all children of the directory and is generating
inotify watch descriptors for all children. A subsequent inotify event
is generated for a subdirectory, but an inotify watch descriptor already
exists from the previous discovery.
In any case, this leads to an out-of-bounds array access on the internal
data structure "wdlt" of clamonacc while trying to look up the "path"
from there. This in turn can lead to a SIGSEGV of the clamonacc-ddd
thread.
The clamonacc-ddd thread (clamonacc/inotif/inotif.c) has currently no
explicit handling for the special inotify kernel events IN_UNMOUNT,
IN_Q_OVERFLOW and IN_IGNORED. It treats the inotify_event structs in
the same way as regular inotify events, although for those special
events the "len" and "name" fields a value of zero respectively NULL.
This can lead to a SIGSEGV of the clamonacc-ddd thread when calling
"strlen(name)" on a NULL value.
This commit adds just a quick'n'dirty workaround for the SIGSEGV and
some logging output. It DOES NOT fix the more overall issue, that a
IN_Q_OVERFLOW inotify event also leads to an out-of-sync situation
between the monitored filesystem/directory and the internal data
structures "ddd_ht" and "wdlt" of clamonacc.
The clamonacc-ddd thread (clamonacc/inotif/inotif.c) currently and in-
discriminately stops processing of inotify watch descriptors in the
function "onas_ddd_unwatch_hierarchy" upon any error returned from the
call to "inotify_rm_watch". This in turn also prevents further updates
and cleanup of the internal data structures "ddd_ht" and "wdlt", which
leads to a invalid value of the "path" variable.
This causes a leak of inotify watch descriptors in the case of e.g. a
move of a directory containing a hierarchy of watched subdirectories.
See the inotify watch descriptor ("wd:") values in the section "Example
and reproducer" below.
This commit adds a workaround for this issue by specifically ignoring
the "ENOENT" errno in case the call to "inotify_rm_watch" results in
a non-zero return code. This allows the cleanup to properly proceed
for all children.
Example and reproducer:
- Starting point:
Log:
ClamInotif: created watch descriptor (wd:1) for path: /clamonacc/monitored
[...]
ClamInotif: created watch descriptor (wd:129637) for path: /clamonacc/monitored/[...]
- Initial directory and subdirectory creation:
mkdir -p /clamonacc/monitored/.test_ff/{1,2,3,4,5}
Log:
ClamInotif: inotify event wd:1, mask:1073742080, cookie:0, length:16, child:.test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: CREATE - adding /clamonacc/monitored/.test_ff to /clamonacc/monitored with wd:1
ClamInotif: created watch descriptor (wd:129638) for path: /clamonacc/monitored/.test_ff
ClamInotif: created watch descriptor (wd:129639) for path: /clamonacc/monitored/.test_ff/3
ClamInotif: created watch descriptor (wd:129640) for path: /clamonacc/monitored/.test_ff/4
ClamInotif: created watch descriptor (wd:129641) for path: /clamonacc/monitored/.test_ff/1
ClamInotif: created watch descriptor (wd:129642) for path: /clamonacc/monitored/.test_ff/5
ClamInotif: created watch descriptor (wd:129643) for path: /clamonacc/monitored/.test_ff/2
- Move / rename of the parent directory:
mv -i /clamonacc/monitored/.test_ff/ /clamonacc/monitored/test_ff
Log:
ClamInotif: inotify event wd:1, mask:1073741888, cookie:12094045, length:16, child:.test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: MOVED_FROM - removing /clamonacc/monitored/.test_ff from /clamonacc/monitored with wd:1
ERROR: ClamInotif: error removing watch descriptor (wd:129638) for path: /clamonacc/monitored/.test_ff, No such file or directory
ClamInotif: inotify event wd:1, mask:1073741952, cookie:12094045, length:16, child:test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: MOVED_TO - adding /clamonacc/monitored/test_ff to /clamonacc/monitored with wd:1
ClamInotif: created watch descriptor (wd:129644) for path: /clamonacc/monitored/test_ff
ClamInotif: created watch descriptor (wd:129639) for path: /clamonacc/monitored/test_ff/3
ClamInotif: created watch descriptor (wd:129640) for path: /clamonacc/monitored/test_ff/4
ClamInotif: created watch descriptor (wd:129641) for path: /clamonacc/monitored/test_ff/1
ClamInotif: created watch descriptor (wd:129642) for path: /clamonacc/monitored/test_ff/5
ClamInotif: created watch descriptor (wd:129643) for path: /clamonacc/monitored/test_ff/2
- Removal of the parent directory unter the new name:
rm -r /clamonacc/monitored/test_ff
Log:
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:3
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/3 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129639) for path: /clamonacc/monitored/test_ff/3
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:4
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/4 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129640) for path: /clamonacc/monitored/test_ff/4
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:1
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/1 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129641) for path: /clamonacc/monitored/test_ff/1
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:5
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/5 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129642) for path: /clamonacc/monitored/test_ff/5
ClamInotif: inotify event wd:129644, mask:1073742336, cookie:0, length:16, child:2
ClamInotif: inotify event path: /clamonacc/monitored/test_ff
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff/2 from /clamonacc/monitored/test_ff with wd:129644
ClamInotif: removed watch descriptor (wd:129643) for path: /clamonacc/monitored/test_ff/2
ClamInotif: inotify event wd:1, mask:1073742336, cookie:0, length:16, child:test_ff
ClamInotif: inotify event path: /clamonacc/monitored
ClamInotif: DELETE - removing /clamonacc/monitored/test_ff from /clamonacc/monitored with wd:1
ClamInotif: removed watch descriptor (wd:129644) for path: /clamonacc/monitored/test_ff
This is also not strictly necessary, but i found it rather helpful to
be able to identify and distinguish the individual clamonacc threads in
the usual Linux system tools ("ps", "top", "gdb", etc.).
Setting the thread name is already done for the threads started by the
bundled c-thread-pool library ("thread-pool-N"). Only the clamonacc
threads all identify themselfes as "clamonacc".
This commit now sets the following - shortened (due to the 16 character
limit) - thread names:
- "clamonacc-ddd" for the ddd/inotify thread.
- "clamonacc-sq" for the scan queue thread.
- the main clamonacc thread currently keeps the name "clamonacc" in-
herited from the main process.
Explicit setting of pthread_sigmask in the individual threads
should not be necessary at all(?), since it is already done in
the main process/thread and should be inherited from there. But
since the sigfillset/sigdelset lines are already there, at least
they should be consistent with regard to the undefined behaviour
caused by ignoring SIGFPE, SIGILL, SIGSEGV, or SIGBUS signals.
* Added loglevel parameter to logg()
* Fix logg and mprintf internals with new loglevels
* Update all logg calls to set loglevel
* Update all mprintf calls to set loglevel
* Fix hidden logg calls
* Executed clam-format
The named "shared" is confusing, especially now that these features are
built as a static library instead of being directly compiled into the
various applications.
- 192959 Resource leak - In cli_bcomp_compare_check: Leak of
memory or pointers to system resources. Several fail cases
could lead to `buffer` or `tmp_buffer` being leaked
- 192934 Resource leak - In cli_bcomp_normalize_buffer: Leak of
memory or pointers to system resources. `hex_buffer` leaked
under certain conditions
- 185977 Resource leak - In ole2_process_property: Leak of memory
or pointers to system resources. A fail case could lead to
`outstr` and `outstr2` being leaked
- 185941 Resource leak - In header_cb (clamsubmit): Leak of
memory or pointers to system resources. A fail case could lead
to `mem` being leaked
- 185925 Resource leak - In load_oneyara: Leak of memory or
pointers to system resources. Several fail cases could lead
to `newident` being leaked
- 185918 Resource leak - In parsehwp3_docsummary: Leak of memory
or pointers to system resources. Not actually a leak, but
caused by checking for a condition that can’t occur.
- 185915 Resource leak - In parsehwp3_docinfo: Leak of memory or
pointers to system resources. Not actually a leak, but caused
by checking for a condition that can’t occur.
- 147644 Resource leak - In tcpserver: Leak of memory or pointers
to system resources. A fail case could lead to `info` being leaked
- 147642 Resource leak - In onas_ht_add_hierarchy: Leak of memory
or pointers to system resources. Several fail cases could lead
to `hnode` or `elem` memory leaks
Coverity warnings:
- 293628 Uninitialized pointer read - In reload_db: Reads
an uninitialized pointer or its target. A fail case
could lead to `rldata` being used before initialization
- 293627 Uninitialized pointer read - In reload_th: Reads
an uninitialized pointer or its target. A fail case could
lead to `engine` being used before initialization
- 265483 Uninitialized pointer write - In parseEmailFile:
Write to target of an uninitialized pointer. A fail case
could lead `ret` to be dereferenced and written to
- 265482 Resource leak - In parseEmailFile: Leak of memory
or pointers to system resources. A fail case could lead
to `head` being leaked
- 225221 Resource leak - In onas_get_opt_list: Leak of memory
or pointers to system resources. A fail case could lead to
`opt_list` being leaked
- 225181 Resource leak - In onas_ht_rm_hierarchy: Leak of
memory or pointers to system resources. A fail case could
lead to `prntname` being leaked
- 193874 Resource leak - In cli_genfname: Leak of memory
or pointers to system resources. A fail case could lead
to `sanitized_prefix` being leaked
- 225196 Resource leak - In onas_fan_eloop: Leak of memory
or pointers to system resources. A fail cases could lead
to `event_data` being leaked
Also, I added some unresolved comments regarding clamonacc
functionality, and added a version compatibility check that
is shown in the example code in the `fanotify` man page
Clamonacc was missing scanning files when the system was under load,
due to scanning on file creation, which would occasionally scan before
the file had been written. This pr modifies clamonacc to scan when the file
is closed.
Added logic to inotify and fanotify startup to print an error and skip
watching the clamd TemporaryDirectory if specified, or the default tmp
directory if not specified to prevent users from watching the directory
where clamd will write temp files. In addition, when using inotify (DDD)
it will try to exclude the clamd temp directory in case it was included
by watching a parent directory. This means that users may set
TemporaryDirectory to something like /tmp/clam and then watch /tmp and
clamonacc will automatically ignore /tmp/clam.
ClamOnAcc may crash when a directory tree is deleted while it's being
scanned. This is easy to reproduce by extracting a large tarball in a
watched directory and then deleting the extracted directory before the
scan is complete.
When removing the inotify nodes, the dirname may be NULL causing a
NULL-dereference. It appears that either the addition or removal
somewhere else in the code is leaving behind the inotify node with a
NULL dirname. I was unable to determine where that bug is, but it was
simple enough to fix the crash by adding a NULL-check. I suspect there's
a memory leak as a result, though a test with valgrind couldn't confirm
it because cleanup in the end on shutdown appears to properly clean up
the inotify watch trees.
Fix addresses https://bugzilla.clamav.net/show_bug.cgi?id=12625
This patch adds experimental-quality CMake build tooling.
The libmspack build required a modification to use "" instead of <> for
header #includes. This will hopefully be included in the libmspack
upstream project when adding CMake build tooling to libmspack.
Removed use of libltdl when using CMake.
Flex & Bison are now required to build.
If -DMAINTAINER_MODE, then GPERF is also required, though it currently
doesn't actually do anything. TODO!
I found that the autotools build system was generating the lexer output
but not actually compiling it, instead using previously generated (and
manually renamed) lexer c source. As a consequence, changes to the .l
and .y files weren't making it into the build. To resolve this, I
removed generated flex/bison files and fixed the tooling to use the
freshly generated files. Flex and bison are now required build tools.
On Windows, this adds a dependency on the winflexbison package,
which can be obtained using Chocolatey or may be manually installed.
CMake tooling only has partial support for building with external LLVM
library, and no support for the internal LLVM (to be removed in the
future). I.e. The CMake build currently only supports the bytecode
interpreter.
Many files used include paths relative to the top source directory or
relative to the current project, rather than relative to each build
target. Modern CMake support requires including internal dependency
headers the same way you would external dependency headers (albeit
with "" instead of <>). This meant correcting all header includes to
be relative to the build targets and not relative to the workspace.
For example, ...
```c
include "../libclamav/clamav.h"
include "clamd/clamd_others.h"
```
... becomes:
```c
// libclamav
include "clamav.h"
// clamd
include "clamd_others.h"
```
Fixes header name conflicts by renaming a few of the files.
Converted the "shared" code into a static library, which depends on
libclamav. The ironically named "shared" static library provides
features common to the ClamAV apps which are not required in
libclamav itself and are not intended for use by downstream projects.
This change was required for correct modern CMake practices but was
also required to use the automake "subdir-objects" option.
This eliminates warnings when running autoreconf which, in the next
version of autoconf & automake are likely to break the build.
libclamav used to build in multiple stages where an earlier stage is
a static library containing utils required by the "shared" code.
Linking clamdscan and clamdtop with this libclamav utils static lib
allowed these two apps to function without libclamav. While this is
nice in theory, the practical gains are minimal and it complicates
the build system. As such, the autotools and CMake tooling was
simplified for improved maintainability and this feature was thrown
out. clamdtop and clamdscan now require libclamav to function.
Removed the nopthreads version of the autotools
libclamav_internal_utils static library and added pthread linking to
a couple apps that may have issues building on some platforms without
it, with the intention of removing needless complexity from the
source. Kept the regular version of libclamav_internal_utils.la
though it is no longer used anywhere but in libclamav.
Added an experimental doxygen build option which attempts to build
clamav.h and libfreshclam doxygen html docs.
The CMake build tooling also may build the example program(s), which
isn't a feature in the Autotools build system.
Changed C standard to C90+ due to inline linking issues with socket.h
when linking libfreshclam.so on Linux.
Generate common.rc for win32.
Fix tabs/spaces in shared Makefile.am, and remove vestigial ifndef
from misc.c.
Add CMake files to the automake dist, so users can try the new
CMake tooling w/out having to build from a git clone.
clamonacc changes:
- Renamed FANOTIFY macro to HAVE_SYS_FANOTIFY_H to better match other
similar macros.
- Added a new clamav-clamonacc.service systemd unit file, based on
the work of ChadDevOps & Aaron Brighton.
- Added missing clamonacc man page.
Updates to clamdscan man page, add missing options.
Remove vestigial CL_NOLIBCLAMAV definitions (all apps now use
libclamav).
Rename Windows mspack.dll to libmspack.dll so all ClamAV-built
libraries have the lib-prefix with Visual Studio as with CMake.
Michael Orlitzky