ClamDScan will leak the memory for the scan target filename if using
`--fdpass` or using `--stream`. This commit fixes that leak.
Resolves: https://bugzilla.clamav.net/show_bug.cgi?id=12648
ClamDScan will fail to scan any file after running into an
"ExcludePath" exclusion when using `--fdpass` or `--stream` AND
--multiscan (-m). The issue is because the parallel_callback()
callback function used by file tree walk (ftw) feature returns an
error code for excluded files rather than "success".
Memory for the accidentally-excluded paths for a given directory also
appears to be leaked.
This commit resolves this accidental-abort issue and the memory leak.
There was an additional single file path memory leak when using
`--fdpass` caused by bad error handling in `cli_ftw()`.
This was fixed by removing the confusing ternaries, and using
separate pointers for each filename copy.
ClamDScan with ExcludePath regex may fail to exclude absolute paths
when performing relative scans because the exclude-check function may
match using provided relative path (E.g. `/some/path/../another/path`)
rather than an absolute path (E.g. `/some/path/another/path`).
This issue is resolved by getting the real path at the start of the
scan, eliminating `.` and `..` relative pathing from all filepaths.
TODO 1: In addition to being recursive (bad for stack safety), the
File Tree Walk (FTW) implementation is a spaghetti code and should
be refactored.
TODO 2: ExcludePath will print out "Excluded" for each path that is
excluded when using `--fdpass` or `--stream`, and for each path
directly scanned that is directly excluded. But in a recursive
regular-scan, the "Excluded" message for the those paths is missing.
The named "shared" is confusing, especially now that these features are
built as a static library instead of being directly compiled into the
various applications.
Users have complained about two specific log events that are extremely
verbose in non-critical error conditions:
- clamonacc reports "ERROR: Can't send to clamd: Bad address"
This may occur when small files are created/destroyed before they can
be sent to be scanned. The log message probably should only be
reported in verbose mode.
- clamonacc reports "ClamMisc: $/proc/XXX vanished before UIDs could be
excluded; scanning anyway"
This may occur when a process that accessed a file exits before
clamonacc find out who accessed the file. This is a fairly frequent
occurence. It can still be problematic if `clamd` was the process which
accessed the file (like a clamd temp file if watching /tmp), generally
it's not an issue and we want to silently scan it anyways.
Also addressed copypaste issue in onas_send_stream() wherein fd is set
to 0 (aka STDIN) if the provided fd == 0 (should've been -1 for invalid
FD) and if filename == NULL. In fact clamonacc never scans STDIN so the
scan should fail if filename == NULL and the provided FD is invalid
(-1).
I also found that "Access denied. ERROR" is easily provoked when using
--fdpass or --stream using this simple script:
for i in {1..5000}; do echo "blah $i" > tmp-$i && rm tmp-$i; done
Clamdscan does not allow for scans to fail quietly because the file does
not exist, but for clamonacc it's a common thing and we don't want to
output an error. To solve this, I changed it so a return length of -1
will still result in an "internal error" message but return len 0
failures will be silently ignored.
I've added a static variable to onas_client_scan() that keeps state in
case clamd is stopped and started - that way it won't print an error
message for every event when offline. Instead it will log an error for
the first connection failure, and log again when the connection is
re-established for a future scan. Calls to onas_client_scan() are
already wrapped with the onas_scan_lock mutex so the static variable
should be safe.
Finally, there were a couple of error responses from clamd that can
occur if the file isn't found which we want to silently ignore, so I've
tweaked the code which checks for specific error messages to account for
these.
Default --wait timeout adjusted from 29 to 30 seconds.
--ping and --wait should exit with CL_ETIMEOUT (21) on timeout.
--ping should only return exit code 0 if clamd responds.
Silenced a couple switch fall-through warnings.
Added proc_fd_fname stack buffer to use with readlink, because the
pointers are restricted and using the same buffer with readlink could
result in undefined behavior.
Relocated clamonacc log verbosity initialization so early verbose log
messages will be printed.
Added a new status code for clamonacc startup checks so the --ping
feature can exit the process early with exit code 0.
Ping interval was not validated properly, causing a crash when the colon
separator was not present between attempts and interval. This was present
in clamonacc, as well as clamdscan.
This patch adds experimental-quality CMake build tooling.
The libmspack build required a modification to use "" instead of <> for
header #includes. This will hopefully be included in the libmspack
upstream project when adding CMake build tooling to libmspack.
Removed use of libltdl when using CMake.
Flex & Bison are now required to build.
If -DMAINTAINER_MODE, then GPERF is also required, though it currently
doesn't actually do anything. TODO!
I found that the autotools build system was generating the lexer output
but not actually compiling it, instead using previously generated (and
manually renamed) lexer c source. As a consequence, changes to the .l
and .y files weren't making it into the build. To resolve this, I
removed generated flex/bison files and fixed the tooling to use the
freshly generated files. Flex and bison are now required build tools.
On Windows, this adds a dependency on the winflexbison package,
which can be obtained using Chocolatey or may be manually installed.
CMake tooling only has partial support for building with external LLVM
library, and no support for the internal LLVM (to be removed in the
future). I.e. The CMake build currently only supports the bytecode
interpreter.
Many files used include paths relative to the top source directory or
relative to the current project, rather than relative to each build
target. Modern CMake support requires including internal dependency
headers the same way you would external dependency headers (albeit
with "" instead of <>). This meant correcting all header includes to
be relative to the build targets and not relative to the workspace.
For example, ...
```c
include "../libclamav/clamav.h"
include "clamd/clamd_others.h"
```
... becomes:
```c
// libclamav
include "clamav.h"
// clamd
include "clamd_others.h"
```
Fixes header name conflicts by renaming a few of the files.
Converted the "shared" code into a static library, which depends on
libclamav. The ironically named "shared" static library provides
features common to the ClamAV apps which are not required in
libclamav itself and are not intended for use by downstream projects.
This change was required for correct modern CMake practices but was
also required to use the automake "subdir-objects" option.
This eliminates warnings when running autoreconf which, in the next
version of autoconf & automake are likely to break the build.
libclamav used to build in multiple stages where an earlier stage is
a static library containing utils required by the "shared" code.
Linking clamdscan and clamdtop with this libclamav utils static lib
allowed these two apps to function without libclamav. While this is
nice in theory, the practical gains are minimal and it complicates
the build system. As such, the autotools and CMake tooling was
simplified for improved maintainability and this feature was thrown
out. clamdtop and clamdscan now require libclamav to function.
Removed the nopthreads version of the autotools
libclamav_internal_utils static library and added pthread linking to
a couple apps that may have issues building on some platforms without
it, with the intention of removing needless complexity from the
source. Kept the regular version of libclamav_internal_utils.la
though it is no longer used anywhere but in libclamav.
Added an experimental doxygen build option which attempts to build
clamav.h and libfreshclam doxygen html docs.
The CMake build tooling also may build the example program(s), which
isn't a feature in the Autotools build system.
Changed C standard to C90+ due to inline linking issues with socket.h
when linking libfreshclam.so on Linux.
Generate common.rc for win32.
Fix tabs/spaces in shared Makefile.am, and remove vestigial ifndef
from misc.c.
Add CMake files to the automake dist, so users can try the new
CMake tooling w/out having to build from a git clone.
clamonacc changes:
- Renamed FANOTIFY macro to HAVE_SYS_FANOTIFY_H to better match other
similar macros.
- Added a new clamav-clamonacc.service systemd unit file, based on
the work of ChadDevOps & Aaron Brighton.
- Added missing clamonacc man page.
Updates to clamdscan man page, add missing options.
Remove vestigial CL_NOLIBCLAMAV definitions (all apps now use
libclamav).
Rename Windows mspack.dll to libmspack.dll so all ClamAV-built
libraries have the lib-prefix with Visual Studio as with CMake.
Add missing ping_clamd() declaration in client.h
Fix check for ping option to first check if ping option is NULL before
strdup'ing and checking if the alloc failed.
Fix format string for uint64_t print.
Correctly assign name pointer to stack buffer in cpio parser.
Remove vestigial variables from insert_list() function matcher-ac.c,
left over from before the load-time optimizations completely
restructured everything.
Silence warnings about unused parameters in progress bar callback
function.
Micah Snyder