This is a bit of a style thing, as freeing a NULL pointer isn't
dangerous. Still, we hould only try to free the certificate name
if non-NULL, and then set it to NULL so it may not be accidentally
re-used if the code is later modified.
The iconv_open() function fails on Alpine because the musl iconv
implementation does not support //TRANSLIT.
See: https://github.com/akrennmair/newsbeuter/issues/364#issuecomment-250208235
This commit also makes it so `ctest` must pass when building the Docker
image, now that the tests with utf8 conversion are expected to pass.
Disable the HTTPUserAgent config option if the DatabaseMirror uses
clamav.net. This will prevent users from being inadvertently blocked and
will ensure that we can keep better metrics on which clamav versions are
being used.
This change is needed because we observed some users being blocked by
the new CDN rules when they set custom user-agents.
The FreshClam DAT file may be owned by root if run like so:
sudo freshclam -d
This is problematic because any future attempts to use FreshClam will
fail unless run as root/with sudo.
This commit fixes the issue by changing the file ownership if it exists
to the DatabaseOwner before dropping privileges. This has the benefit of
changing any previously existing files to be owned by the DatabaseOwner
for folks who were already affected by the bug.
Some users have scripts set up from long ago to delete mirrors.dat if
FreshClam failed. We used to recommend this if people had technical
issues because mirrors.dat would store a bunch of entries indicating
that all of their regional mirrors were failing and then FreshClam would
give up.
The new freshclam DAT file no longer stores that kind of information.
Deleting the DAT file is no longer sound advice.
We very much want the UUID, which is generated when creating the DAT
file, to persist between runs. So unless people go and change the
scripts to delete freshclam.dat instead, this commit should resolve the
concern.
Improvements to use modern block list and allow list verbiage.
blacklist -> block list
whitelist -> allow listed
blacklisted -> blocked
whitelisted -> allowed
In the case of certificate verification, use "trust" or "verify" when
something is allowed.
Also changed domainlist -> domain list (or DomainList) to match.
The subject and public key of each CRB signature used to be
printed in debug mode, but this will become increasingly
more noisy as we add more CRB rules. For now, given that the
number of CRB sigs is still low, only print this information
when running clamscan with the --dumpcerts parameter (or
equivalent clamd setting). If we ever get to the point where
we have hundreds of CRB rules and these log messages are
overwhelming when running with --dumpcerts, consider removing
the log messages entirely and update sigtool so that it can
nicely format and display CRB sigs with using the --decode
parameter.
clamav-daemon.service requires clamav-daemon.socket but
it is missing from the build and install process.
Signed-off-by: Armin Kuster <akuster808@gmail.com>
We used to generate the tarball in the build pipeline, but that required
cloning the source repo a second time in the build pipeline.
The top-level pipeline will now generate the tarball and hands it off
to downstream pipelines, that way the build pipeline will work w/out
having to know which git repository to clone.
The end goal being that our internal QA suite can trigger off of the
public git repo or an internal git repo.
Have to manually link libtinfo (`-ltinfo`) because our FindLLVM
didn't add it to the LLVM_LIBRARIES variable for us. See:
- https://stackoverflow.com/questions/21477407/llvm-3-5-fails-to-link
Have to remove the CXX_STANDARD setting at the top of CMakeLists.txt
because of c++90 / c++11 ABI compatibility issues w/ LLVM. See:
- https://maleadt.github.io/LLVM.jl/dev/man/troubleshooting/
Rename "llvm/Config/config.h" "llvm/Config/llvm-config.h" because
LLVM renamed it in 2.8.
Have to link LLVM manually with the test binaries that use the
clamav object library instead of libclamav shared library.
CMake does not propagate library dependencies from object files.
I tested on ubuntu:16.04 with LLVM 3.6 built from source using:
```
/usr/local/bin/cmake .. -D CMAKE_INSTALL_PREFIX=/opt/llvm/3.6 \
-D LLVM_ENABLE_RTTI=ON
```
Then built clamav w/:
```
/usr/local/bin/cmake .. -D CMAKE_INSTALL_PREFIX=`pwd`/install \
-D BYTECODE_RUNTIME="llvm" \
-D LLVM_ROOT_DIR="/opt/llvm/3.6" \
-D LLVM_FIND_VERSION="3.6.0" && make && make install
```
CMake is now required to build.
The built-in LLVM is no longer available.
Also removed support for libltdl calls, which is not used in the CMake
builds, was only used when building with Autotools.
TODO: Fix CMake LLVM support & update to work with modern versions.
If you test in a docker container and don't create a user/switch to run
as something other than root, than the tests will fail.
Default freshclam behavior is to drop to run as the "clamav" user when
run as root. This change sets the DatabaseOwner to the current user, so
it the tests will pass when run as root.
Add options to build a source tarball.
This will replace Autotools' `make dist` feature.
Although CMake doesn't have an autogen step, this is still necessary if
we want to bundle in a local copy of the online documentation or if we
want to vendor dependencies from github/etc at configure time in the tarball.
There is a scan logic issue where the main libclamav scanning functions
create an extra "nested" fmap for each file being scanned. This is
slightly inefficient for a normal scan, but causes a major performance
issue when using ENGINE_OPTIONS_FORCE_TO_DISK. It causes every scanned
file to be duplicated in the temp directory before the scan.
We fix this by using `cli_magic_scan()` in `scan_common()` instead
of `cli_magic_scan_nested_fmap_type()`. We can do this now that the
`cl_scandesc_callback()` API creates an fmap for the caller, instead of
the old logic where `scan_common()` called different API's depending on
whether or not we have an fmap or a file descriptor.
I'd foolishly set the Iconv detection module to use uppercase to
differentiate it from the upstream (original) module. I also kind of
forgot to change the documentation to use the uppercase name. In
reviewing that, I decided it's better to match the upstream
FindIconv.cmake module as closely as possible in case they fix the issue
described here, so that we can stop carrying a custom module without
breaking the CMake options for everyone:
b26cdc5453
This reverts the CMake options for setting the Iconv include and library
paths back to match the upstream before 0.104 ships with stable CMake
support.
The WebServerHandler classes used for the tests have a "Test_" prefix
which causes pytest to print some warnings thinking they're special test
classes. This commit gets rid of the warnings.
If pytest is available in the path, but wasn't found in the Python 3
installation discovered by CMake, then the test suite will fail.
This can happen when two different Python 3 installations exist, eg 3.6
and 3.7 and when the newest one (prefered by CMake) doesn't have pytest.
This commit will use the correct command depending on where pytest was
(or wasn't) found. It will also fail more gracefully if neither pytest
nor unittest was found (this happens with some wierd python installs).
The milter default was not set correctly so it was starting
clamav-milter by default. Added default true/false settings for each of
the docker daemon environment variables.
Added command to entrypoint to change database directory ownership to
the clamav user, in case it is a mounted volume (which get root ownership
by default).
Removed the clamav user & group from the build image, as it isn't needed
until the base image.
Added announcement details and acknowledgements to the News document.
Added details to the Docker Readme describing:
- the tag naming convention
- how to mount database volumes
- various other tweaks
Fixed the path for the update script in the GitHub Action, and changed
from alpine-latest to ubuntu-latest, because alpine-latest doesn't seem
to be an option.
To reduce the need for constant database updates, also push a docker
image containing the virus database.
Ideally, this script is called every time a major virus database update
happens, but as that requires integration with other systems, lets start
by having a scheduled update.
To do this a github action workflow was added on a schedule that runs
once a month. It does require the proper secrets to be setup however and
made accessible.
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
This dockerfile allows users/services compile the current codebase and
use the container as a dockerized clamav daemon.
Useful for testing, but also for running clamd containerized.
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
As pytest can be installed in many ways, we can check for pytests
existence next to if it was pip-installed.
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
There are some merits of having the configs in `/etc/clamav` and the databases
in `/var/lib/clamav` to match the way the distros all package it and so these
directories can be mounted volumes if needed.
The DATABASE_DIRECTORY and APP_CONFIG_DIRECTORY variables don't support
absolute paths. This patch changes that.
The testcase.py script is using Python 3.6+ syntax for the NamedTuple.
This commit reverts to the old 3.5 syntax until we can drop support for
Python 3.5.
The named "shared" is confusing, especially now that these features are
built as a static library instead of being directly compiled into the
various applications.
The previous UnRAR vuln fix worked for the (old, gone) Visual Studio
solution but broke the CMake build. We didn't notice because we tested
and approved it with Visual Studio before switching to CMake.
This change switches it to use the correct macro for the
libclamunrar_iface extension which is ".dll" instead of "lib".
The size of the UE buffer for the new Adobe Reader X encryption support
was not properly recorded and may result in reading too far into the UE
buffer. This patch checks the size of the UE buffer and rejects it if
the length is not 32, as it does with the other AES256 CBC method.
The XLM formula parser failed to account for string records that claim
to be longer than the formula data. This fix skips over the invalid
string records.
Also fixed an unrelated XLM parsing bug where BIFF name records weren't
handled on builds lacking the json-c library, resulting in verbose error
output. See https://bugzilla.clamav.net/show_bug.cgi?id=12639
After the previous improvements, there is still an issue wherein a
private database mirror may not yet have the very latest CVD and CDIFF.
In this case, and if the clients are still using the official DNS server
to check the latest versions, then the clients will try several times to
update with CDIFFs before downloading the entire CVD. This is clearly
not desirable, especially because that CVD is probably also 1 version
older than expected.
This commit keeps track of the number of patches and will only ask for
the CVD after the patches failed if no patches were downloaded AND we're
more than 1 version behind. What that means is that if we're 1 version
behind and the server is too -- it will log some warnings but succeed.
Or if we are multple versions behind but we did get some patches, we'll
settle for a minor update.
This change will essentially allow FreshClam to be used with private
mirrors that aren't updated the second a new daily database is
published. Without this change, private mirror operators may end up
frequently serving the entire daily CVD to fussy FreshClam clients.
Micah Snyder (micasnyd)