This improvement looks up the filename given the file descriptor.
This is supported on Mac and Linux but not presently supported
on other UNIX operating systems. FD-passing is not available on
Windows.
On supported systems, the verdict in the clamd log and the VirusEvent
will show the actual file path instead of something like fd[14].
Added logic to inotify and fanotify startup to print an error and skip
watching the clamd TemporaryDirectory if specified, or the default tmp
directory if not specified to prevent users from watching the directory
where clamd will write temp files. In addition, when using inotify (DDD)
it will try to exclude the clamd temp directory in case it was included
by watching a parent directory. This means that users may set
TemporaryDirectory to something like /tmp/clam and then watch /tmp and
clamonacc will automatically ignore /tmp/clam.
Users have complained about two specific log events that are extremely
verbose in non-critical error conditions:
- clamonacc reports "ERROR: Can't send to clamd: Bad address"
This may occur when small files are created/destroyed before they can
be sent to be scanned. The log message probably should only be
reported in verbose mode.
- clamonacc reports "ClamMisc: $/proc/XXX vanished before UIDs could be
excluded; scanning anyway"
This may occur when a process that accessed a file exits before
clamonacc find out who accessed the file. This is a fairly frequent
occurence. It can still be problematic if `clamd` was the process which
accessed the file (like a clamd temp file if watching /tmp), generally
it's not an issue and we want to silently scan it anyways.
Also addressed copypaste issue in onas_send_stream() wherein fd is set
to 0 (aka STDIN) if the provided fd == 0 (should've been -1 for invalid
FD) and if filename == NULL. In fact clamonacc never scans STDIN so the
scan should fail if filename == NULL and the provided FD is invalid
(-1).
I also found that "Access denied. ERROR" is easily provoked when using
--fdpass or --stream using this simple script:
for i in {1..5000}; do echo "blah $i" > tmp-$i && rm tmp-$i; done
Clamdscan does not allow for scans to fail quietly because the file does
not exist, but for clamonacc it's a common thing and we don't want to
output an error. To solve this, I changed it so a return length of -1
will still result in an "internal error" message but return len 0
failures will be silently ignored.
I've added a static variable to onas_client_scan() that keeps state in
case clamd is stopped and started - that way it won't print an error
message for every event when offline. Instead it will log an error for
the first connection failure, and log again when the connection is
re-established for a future scan. Calls to onas_client_scan() are
already wrapped with the onas_scan_lock mutex so the static variable
should be safe.
Finally, there were a couple of error responses from clamd that can
occur if the file isn't found which we want to silently ignore, so I've
tweaked the code which checks for specific error messages to account for
these.
clamonacc's --wait option was broken and would exit as soon as clamd
responded, rather than starting clamonacc. The fix is simply to return
"success" when the pong is received, rather than "break".
clamonacc's --watch-list option's short-hand "-w" conflicts with the
--wait option's "-w" short-hand. This causes --watch-list to be
non-functional, invoking the --wait option when you use --watch-list.
This patch switches the --watch-list short-hand to "-W".
ClamOnAcc may crash when a directory tree is deleted while it's being
scanned. This is easy to reproduce by extracting a large tarball in a
watched directory and then deleting the extracted directory before the
scan is complete.
When removing the inotify nodes, the dirname may be NULL causing a
NULL-dereference. It appears that either the addition or removal
somewhere else in the code is leaving behind the inotify node with a
NULL dirname. I was unable to determine where that bug is, but it was
simple enough to fix the crash by adding a NULL-check. I suspect there's
a memory leak as a result, though a test with valgrind couldn't confirm
it because cleanup in the end on shutdown appears to properly clean up
the inotify watch trees.
Fix addresses https://bugzilla.clamav.net/show_bug.cgi?id=12625
When using --on-update-execute=EXIT_1 freshclam doesn't clean up the
temporary directory where it downloaded and tested the new database.
This patch moves the command execution to happen after temp-cleanup.
The security improvement to perform file realpath lookups prior to a
scan has the adverse effect of causing file scans to fail on Windows
when scanning on some filesystems.
Specifically, it was observed that the ImDisk driver doesn't handle the
IRP_MJ_QUERY_INFORMATION message so the call to look up the realpath
using GetFinalPathNameByHandleW() doesn't work.
There are two other API's I've found which can query the real file path.
The first is to create a file mapping of the target file and then use
GetMappedFileNameW() to get the file path. The other is to use the
NtQueryObject() undocumented NT API to get the file path. Each of
these should return roughly the same thing. For files in an ImDisk
RAM-disk drive, the resulting filepath for R:\clam.exe would
be \\Device\ImDisk0\clam.exe. The trouble is, mapping
\\Device\ImDisk0\clam.exe back to R:\clam.exe would rely on an
assumption that ImDisk is using the default drive letter, which is a tad
hacky.
Instead, this patch simply allows the scan to proceed if the realpath
lookup failed. If the user is using the quarantine (remove/move)
features AND if the scan target filepath has a directory junction (soft
link), then the quarantine action will fail. It's not ideal but it is
quite unlikely.
Move from using curl when attempting to pass file descriptors to
using system calls
System calls must be used here since the kernel translates file
descriptors from one process to another internally when passed
Changing umask for PID files from `0002` to `0022` to limit write access
to the current user only, removing group write access.
Changing directory permissions for the freshclam tmp directories from
`0755` to `0700` because only the freshclam user needs access to these
directories.
Patch courtesy of Tuomo Soini
The linker is unable to find libncurses.so.6.2 on FreeBSD.
If I change Curses::curses from an UNKNOWN library to an
INTERFACE library, then it works fine. While UNKNOWN
appears to be preferred by the CMake community, it seems
like it doesn't add -L/usr/local/lib to the LDFLAGS on my
FreeBSD test machine where /usr/local isn't used by
default.
Disclaimer: This fix feels like a bit of a hack, but it works for me.
On systems with both libiconv and built-in iconv (libc), the compile
test must include the libiconv header path because it _will_ fail if
it builds against libiconv's iconv.h and doesn't link with libiconv.
This fix is similar to the one for Snort3, here:
https://github.com/snort3/snort3/issues/62
Remove the "-rc2" from the version string.
Also bump FLEVEL from 120 -> 121.
Also fixes two issues:
- The VERSION_SUFFIX defined by clamav-config.h.cmake.in must be defined
with #define instead of #cmakedefine, so it is defined as an empty
string even if there is no suffix (eg for an actual release)
- Removed a bashism in the libcheck detection code for autotools,
resolving https://bugzilla.clamav.net/show_bug.cgi?id=12598
At least some unicode filenames may fail to scan in 0.102.4+ because
while Windows char* strings may be UTF8, the GetFinalPathNameByHandleA
function does not return UTF8 strings and instead does lossy conversion
to ASCII. To fix this, we need to use GetFinalPathNameByHandleW instead
and then convert from UTF16-LE to UTF8.
While fixing this bug, I found and fixed a couple other serious issues
with the Win32 implementation of cli_codepage_to_utf8().
If a file is on a network share, the realpath comes back with a path
name that looks like "\\\\?\\UNC\\<host>\\<share>\\...". In thi scase,
the "\\\\?\\UNC\\" prefix is critical or else clamscan.exe won't be able
to open the file. This patch checks for the "\\\\?\\UNC" prefix and if
it exists, it keeps the prefix, else it trims the "\\\\?\\" portion as
before. This should fix scanning of files on network shares.
As libfreshclam links in libshared.a, we don't need to link it again
into freshclam. Presently we do and as a consequence the global
variables in libshared.a cannot be reliably set in freshclam/libfreshlam
and be expected to have the correct values within libshared.so calls.
Specifically, users observed freshclam failing to use syslog even though
syslog was enabled. The reason was that the libshared.a logg() function
will only write to syslog() if the logg_syslog global is non-zero.
When libshared.a is linked twice (first into libfreshclam and again into
freshclam) then setting the logg_syslog global might not set the same
variable that the logg() function depends on.
To address this issue this patch stops linking libshared.a into
freshclam and instead exports the required libshared.a symbols from
libfreshclam.so (using libfreshclam.map) so that freshclam can still use
those functions.
The previous valgrind suppression rule for wcsrtombs assumed ascii
conversion. With the clamd & clamscan change to use setlocale() to get
utf8 filenames to work in UnRAR, the wcsrtombs stack trace changed to
use utf8.
This patch supplements valgrind.supp with the new stack trace. The old
one was left in, just in case.
Fixes error handling issues in ARJ parser wherein FALSE is mistakenly
returned instead of a CL_E* error code, as the type is return type is
`int`, but in reality a cl_error_t enum value is expected.
The UnRAR library requires the character-classification locale to be set
to the empty string "" so it will be set according to the environment
variables, as seeen in the rar.cpp example application `main()`.
Without this, extracting RAR archives containing unicode filenames on
non-Windows, non-macOS operating systems may fail.
Flex and Bison are generally available not not particularly easy to
install and on macOS, the Bison version is relatively ancient and not
compatible. Homebrew doesn't necessarily play nice with Xcode, so to
make CMake builds work on macOS without mandating the use of Homebrew,
our best option is to make Flex & Bison optional.
Flex and Bison generated files will be kept in revision control and will
get re-generated only if you use -DMAINTAINER_MODE=ON which will
introduce the Flex and Bison tool dependencies.
CMake: don't emit fullpath in yara generated source
Autotool's ylwrap script has a hack that prevents the full path of the
bison & flex generated source from being included in the debug line
numbers and in the preprocessor include guard macros. CMake doesn't have
this, so when it sets the output file to the full path, the current
user's path is leaked into the generated source.
Added `%output "yara_grammar.c"` to yara_grammar.y and re-generated the
.c & .h file with this change. This overrides the "FILE" setting used
when generating those line numbers and include guard macro names so that
the path isn't included.
Similarly, added `%option outfile="yara_lexer.c` to yara_lexer.l and
re-generated the .c file with this change. This has the same effect but
for flex so that full filepaths are not emitted into the source.
Revert NEWS.md item regarding Flex, Bison change.
Revert placing yara grammar/lexer files in win32 compat.
Drop privileges in the parent process before waiting for the signal
from the child process, so that the parent properly responds to
the signal and terminates.
Verify that the log file will be owned by user that the deamon will
run as.
Explicitly set PID file ownership to root when starting the daemon
as root with the PID file enabled.
To fix an old (and probably no longer manifest) build bug, this commit
adds the ability to detect an 8-bit libpcre2 with pkg-config in
addition to the other methods of specifying/detecting it. The
intention is that pkg-config will be used only in the default case,
where pcre support is detected automatically; that is, NOT in either
of these cases:
* --with-pcre=<path> is given
* --with-pcre=no is given
The code in pcre.m4 was modified minimally so that --with-pcre=yes
(the default) tries pkg-config first, and then falls back to whatever
it used to do. If pkg-config can find the library, we add PCRE_LIBS
to the LIBS variable and we're done. Otherwise, the old behavior is
retained.
ClamAV-bug: https://bugzilla.clamav.net/show_bug.cgi?id=12484
Gentoo-bug: https://bugs.gentoo.org/567680
This commit also collects the preprocessor flags obtained from
pkg-config.
When libpcre2 is found using pkg-config, we now say that explicitly,
and include its PCRE_LIBS and PCRE_CFLAGS in the output.
clamscan.exe fails to delete a file from C:\somefolder though it can
successfully delete a file from a user-owned directory (eg
C:\Users\user\somefolder). From what I can tell, this is because a new
directory in C:\somefolder does not have the full-access ACL by
default.
This patch changes the NtCreateFile() call to open with generic read
& delete rights, instead of requesting full-access.
The pcre2.h header dependency is propagated to the bytecode runtime,
lzma_sdk, yara, and regex build targets within the libclamav build
because it is included by matcher.h which is included all over the
place.
This patch adds the pcre2 dependency to the affected build targets so
that systems where pcre2 isn't in the standard include path can still
build.
Also removed CMake `PCRE2_DIR` from documentation, as it doesn't apply
to this PCRE2 detection logic that we settled on.
There is an autotools variable and clamav-config.h macro BUILD_CLAMD
which dates back 17 years to a time where libpthread support was
optional and users could build clamscan without building clamd,
clamdscan, clamav-milter, etc. We don't live in that world anymore and
certainly don't test that configuration.
To get the NotifyClamd freshclam feature to work with CMake builds we
either have to define BUILD_CLAMD from CMake as well, or remove the
feature entirely. I chose to remove it, because we don't test builds
without BUILD_CLAMD enabled and it adds needless complexity to our build
system and source.
If libcheck is missing, the unit tests are skipped. Header include path
changes mean clamav.h is not available when the unit tests are skipped,
and thus the check_clamav_skip.c files won't compile. This patch
removes the clamav.h dependency so `make check` will "work".
Default --wait timeout adjusted from 29 to 30 seconds.
--ping and --wait should exit with CL_ETIMEOUT (21) on timeout.
--ping should only return exit code 0 if clamd responds.
Silenced a couple switch fall-through warnings.
Added proc_fd_fname stack buffer to use with readlink, because the
pointers are restricted and using the same buffer with readlink could
result in undefined behavior.
Relocated clamonacc log verbosity initialization so early verbose log
messages will be printed.
Added a new status code for clamonacc startup checks so the --ping
feature can exit the process early with exit code 0.
Update the NEWS to add and correct content prior to the release
candidate.
Changed the version string to have the -rc suffix.
Also fixed a couple of --help and manpage issues.
Ping interval was not validated properly, causing a crash when the colon
separator was not present between attempts and interval. This was present
in clamonacc, as well as clamdscan.
This patch adds experimental-quality CMake build tooling.
The libmspack build required a modification to use "" instead of <> for
header #includes. This will hopefully be included in the libmspack
upstream project when adding CMake build tooling to libmspack.
Removed use of libltdl when using CMake.
Flex & Bison are now required to build.
If -DMAINTAINER_MODE, then GPERF is also required, though it currently
doesn't actually do anything. TODO!
I found that the autotools build system was generating the lexer output
but not actually compiling it, instead using previously generated (and
manually renamed) lexer c source. As a consequence, changes to the .l
and .y files weren't making it into the build. To resolve this, I
removed generated flex/bison files and fixed the tooling to use the
freshly generated files. Flex and bison are now required build tools.
On Windows, this adds a dependency on the winflexbison package,
which can be obtained using Chocolatey or may be manually installed.
CMake tooling only has partial support for building with external LLVM
library, and no support for the internal LLVM (to be removed in the
future). I.e. The CMake build currently only supports the bytecode
interpreter.
Many files used include paths relative to the top source directory or
relative to the current project, rather than relative to each build
target. Modern CMake support requires including internal dependency
headers the same way you would external dependency headers (albeit
with "" instead of <>). This meant correcting all header includes to
be relative to the build targets and not relative to the workspace.
For example, ...
```c
include "../libclamav/clamav.h"
include "clamd/clamd_others.h"
```
... becomes:
```c
// libclamav
include "clamav.h"
// clamd
include "clamd_others.h"
```
Fixes header name conflicts by renaming a few of the files.
Converted the "shared" code into a static library, which depends on
libclamav. The ironically named "shared" static library provides
features common to the ClamAV apps which are not required in
libclamav itself and are not intended for use by downstream projects.
This change was required for correct modern CMake practices but was
also required to use the automake "subdir-objects" option.
This eliminates warnings when running autoreconf which, in the next
version of autoconf & automake are likely to break the build.
libclamav used to build in multiple stages where an earlier stage is
a static library containing utils required by the "shared" code.
Linking clamdscan and clamdtop with this libclamav utils static lib
allowed these two apps to function without libclamav. While this is
nice in theory, the practical gains are minimal and it complicates
the build system. As such, the autotools and CMake tooling was
simplified for improved maintainability and this feature was thrown
out. clamdtop and clamdscan now require libclamav to function.
Removed the nopthreads version of the autotools
libclamav_internal_utils static library and added pthread linking to
a couple apps that may have issues building on some platforms without
it, with the intention of removing needless complexity from the
source. Kept the regular version of libclamav_internal_utils.la
though it is no longer used anywhere but in libclamav.
Added an experimental doxygen build option which attempts to build
clamav.h and libfreshclam doxygen html docs.
The CMake build tooling also may build the example program(s), which
isn't a feature in the Autotools build system.
Changed C standard to C90+ due to inline linking issues with socket.h
when linking libfreshclam.so on Linux.
Generate common.rc for win32.
Fix tabs/spaces in shared Makefile.am, and remove vestigial ifndef
from misc.c.
Add CMake files to the automake dist, so users can try the new
CMake tooling w/out having to build from a git clone.
clamonacc changes:
- Renamed FANOTIFY macro to HAVE_SYS_FANOTIFY_H to better match other
similar macros.
- Added a new clamav-clamonacc.service systemd unit file, based on
the work of ChadDevOps & Aaron Brighton.
- Added missing clamonacc man page.
Updates to clamdscan man page, add missing options.
Remove vestigial CL_NOLIBCLAMAV definitions (all apps now use
libclamav).
Rename Windows mspack.dll to libmspack.dll so all ClamAV-built
libraries have the lib-prefix with Visual Studio as with CMake.
Micah Snyder (micasnyd)