The commit "Freshclam create database directory if missing" inadvertently
broke the build, because it was blindly rebased and merged after a prior
commit relocated the required `statbuf` variable.
This commit adds back the missing `statbuf` variable.
This fixes issues in cvd download when network speed is slow.
Setting is passed to libcurl CURLOPT_TIMEOUT. Original default of 60s
was not enough if network speed is limited. Curl handles this as
total time for http(s) transfer.
https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html
Also change commented out setting of ReceiveTimeout on example configs
to somewhat sensible value (1800s).
Signed-off-by: Tuomo Soini <tis@foobar.fi>
On initialization, freshclam will create the database directory if it is
missing.
If running as root, freshclam will assign ownership of the new directory
to the DatabaseOwner account.
Disable line wrap when printing the progress bar so that small terminal
windows do not see excessive lines printed.
Reduce the number of characters in the progress bar to accomodate
80-char width terminals.
Correctly display number of kilobytes (KiB) in progress bar. Previously
was showing # of MiB but printing "KiB".
Freshclam creates a tmp directory in the database directory used to
store downloaded patches or databases before they replace current
databases. The tmp directory previously was created at when freshclam
was initialized and deleted when freshclam exited. This was problematic
if freshclam was run in daemon mode and then run manually while the
daemon was already running.
This commit alters the behavior to create tmp directory with a random
suffix before the update begins and remove this directory when the
update ends, allowing freshclam to be run manually without causing the
freshclam daemon to fail later.
Users have reported that freshclam will fail to download daily.cvd to an
empty database directory shortly after a new database is published due
to a mirror synchronization error. This may occur if the CDN serves up
an older version while a newer version has just been published. Clearing
the CDN cache has not eliminated the issue.
This commit allows freshclam to accept a database 1 version older than
advertised by DNS but will still fail if the database is more than one
version out of date.
Removing problematic call to convert file descriptors to filepaths.
Added filename and tempfile names to scandesc calls in clamd.
Added a general scan option to treat the scan engine as unprivileged,
meaning that the scan engine will not have read access to the file.
Added check to drop a temp file for RAR's where the we don't have
read access to the filepath provided (i.e. unprivileged is set, or
access() check fails).
Users have reported slow scan speeds of some PDF documents. The scan
speed was very slow on Windows in particular. Investigation indicated
significant time spent in cli_realloc.
Performance is particularly bad with a relatively large data stream is
decompressed using small chunk sizes and the final buffer is reallocated
to a larger buffer size each time a chunk is added.
This commit replaces BUFSIZ, which varies from 256B -> 8192B, with
INFLATE_CHUNK_SIZE, set to 256kB, the chunk size recommended by the zlib
documentation for efficient inflate performance. The output buffer is
shrunk (reallocated) down to the final decoded buffer length so as not
to waste memory when many small buffers must be decompressed.
A followup fix should provide a standard way to do zlib decompression
across libclamav where a linked list of decompressed chunks are
assembled and then the final output buffer is allocated at the end.
Implemented several maximums in parsing MIME messages to avoid Denial of
Service attempts, as well as improving parsing logic to avoid repeatedly
calling the realloc function. These are in response to excessively long scan
times for specially crafted files.
This is in response to CVE-2019-15961.
The limits added are
1. Limit on number of MIME parts per message.
2. Limit on number of header bytes.
3. Limit on number of email headers.
4. Limit on number of line folds.
5. Limit on numbef of MIME arguments.
Tests in libcheck 0.13 must have {} between START_TEST and END_TEST
else it will not compile.
Also replaced all deprecated "fail_" macros with "ck_" macros.
E.g. fail_unless() becomes ck_assert_msg()
The checks_common.h header file provided a couple of macros to
support versions older than 0.9.3. As these older versions are
no longer relevant, I've removed those compatibility macros
entirely.
The ClamAV engine abstraction was migrated into the test-framework
repository to provide a single unified framework for build acceptance
and quality assurance tests. This commit updates the Jenkinsfile
accordingly.
This addresses an issue introduced early in 0.103 development when the
start/end timestamps were added to the scan summary.
The localtime function equivalent on Windows has the argument order reversed.
Some minor changes to clamdtop enabling it to compile on Windows.
Adds clamdtop to the Visual Studio solution and InnoSetup installer.
Clamdtop requires curses, which means that pdcurses will be required
to build ClamAV on Windows. A recipe to build pdcurses has been added
to the clamav-mussels-cookbook repository.
Certs can omit the boolean field in the Basic Constraints section,
since the RFC specifies a default value for this field. This fixes
the following error:
LibClamAV debug: asn1_expect_objtype: expected type 01, got 02
LibClamAV debug: asn1_get_x509: An error occurred when parsing x509 extensions
LibClamAV debug: asn1_parse_mscat: skipping x509 certificate with errors
Ex: 05de45fd6a406dc147a4c8040a54eee947cd6eba02f28c0279ffd1a229e17130
Allow UTCDate fields in x509 certs to omit the seconds. Technically
this is disallowed by RFC5280, but Windows Authenticode verification
routines don't seem to mind it, so we'll allow it too. This fixes
the following error:
LibClamAV debug: asn1_getnum: expecting digits, found 'Z'
LibClamAV debug: asn1_get_time: invalid second 4294967295
LibClamAV debug: asn1_get_x509: unable to extract the notBefore time
LibClamAV debug: asn1_parse_mscat: skipping x509 certificate with errors
Ex: d577010638f208ad8b6dab1a33dc583b2ec6b0c719021fb9f083dd595ede27e8
Also, add a check on CRT_RAWMAXLEN, since if it's > 256 problems
will arise
The previous commit allowed a CRB cert's exponent to be ignored
when evaluating blacklist rules, but this commit also allows
the exponent to be ignored for whitelist rules as well.
Also, previous versions of ClamAV allowed the serial number hash
field in a CRB rule to be blank, effectively wildcarding the serial
number. This functionality broke with some of the changes introduced
in 0.102.0, so this commit addresses that.
CRB rules allow the exponent to be specified, but currently this value
gets ignored and hardcoded to 65537. It turns out that most certs I
tested against (12,000 from VT) use e==65537, but a handful don't.
This commit addresses the signature load time issue in the following steps:
1. Loaded list items are allocated but left unattached; only a node reference is set on them for further processing. This is done with no increase of memory usage. See changes in insert_list and matcher-ac.h
2. Before the tries are built, the whole list of entries is sorted by node, then by pattern, then by partno. This requires O(N log(N)) time.
3. The list is processed linearly, one node at a time and the `next_same` chain is built. Each next_same chain head is also extracted. This requires O(N) time.
4. The list of heads is sorted by partno. This requires O(M log(M)) time on average with M<=N.
5. The list of heads is processed linearly and the `next` chain is built. This has O(M) complexity.
And improves scantime performance, by adding checks to:
1. Place longer lists earlier in the trie.
2. Keep close patterns close, rather than scattering them further apart.
This reduced memory cache faults to improve load and scan time performance.
Bumps the version from 0.102.0 to 0.103.0-devel-<date>.
Bumps the FLEVEL from 111 to 120.
Bumps the libclamav and libfreshclam revision numbers from 4 -> 5, and 0 -> 1, respectively.
Mickey Sola