http://open-source-security-software.net/releases.atom Recent releases 2022-08-16T05:10:56.387808+00:00 python-feedgen reko version-0.11.0 reko version-0.11.0 2022-03-13T23:15:46+00:00 This release of Reko has breaking changes in interfaces and classes, and completes the move to .NET 5.0. Many classes were also moved to different namespaces: you'll need to recompile your project. The file loading code was refactored heavily to be easier to use and to support the reading of files stored in (potentially deeply nested) archives. Many fixes were made in the AArch64 rewriter (with gentle prodding from @rfalke). Some other new features are: * Wasm files can be loaded and disassembled. A rewriter will materialize in a later release. * Reko Gui is asynchronous. This will help the ongoing port to Avalonia. * Stack variable references that escape to other procedures are now tracked (courtesy of @ptomin). * The new ByteTrie<T> class can be used for pattern matching. * Added support for PDP-10 architecture. The PDP-10 is word-addressable, has 36-bit words, 18-bit addresses, and variable sized bytes, all of which contrast vividly with present day 8-bit-based architectures. * The MemoryControl displays sizes other than bytes. This accommodates PDP-10, Microchip PIC, and Mil-Std-1750A. * Intrinsic procedures can now have generic parameter and return types. * Added initial support for COFF files. * Adapted OllyLangInterpreter to other architectures than X86. * Added m6502 emulator, c64 emulator. * Various CI improvements and fixes. Also, Reko now builds on ARM64! (courtesy of @smx-smx) * Added support for constants larger than 64 bits. * Support for reading TAR and AR archives. * Support loading files from arbitrarily nested archives. * Rewrote the loader so it is much easier to use (and understand). * Extended C parser to handle more GCC attributes. * Support for Sanyo LC8670. Thanks to @nemerle, @ptomin, @shandianchengzi, @slartibardfast, and @smx-smx for their time and contributions to the Reko project! 2022-03-13T23:15:46+00:00 AIL-framework v4.1 AIL-framework v4.1 2022-03-14T15:39:04+00:00 # AIL Framework version 4.1 released with new investigation/case handling, improved MISP export and many improvements. ## Investigation in AIL The major new functionality is the investigation handling in AIL. An analyst can now easily create investigation where any objects from AIL can be added. This helps an analyst to build collection or cases to work on. The integration allows to export it as a standard [MISP](https://www.misp-project.org/) event. ![](https://www.ail-project.org/assets/img/ail-investigation.png) ## Support for Jabber/XMPP AIL has been extended to support Jabber/XMPP addresses. The source feeder just need to submit the keys such as `jabber:to`, `jabber:from`, `jabber:ts`, `jabber:id`. An example feeder is [available](https://gist.github.com/gallypette/8e4fc941443a2483b6b2fcaee4c76e47). The new feature can be used to inject existing leak or stream from XMPP/Jabber server. As an example, the Conti leak can be easily injected into AIL and show automatically all correlations between users. ![](https://www.ail-project.org/assets/img/bitcoinz.png) ![](https://www.ail-project.org/assets/img/friends-having-chats.png) ![](https://www.ail-project.org/assets/img/korben.png) Many bugs were fixed. The [complete changelog](https://www.ail-project.org/ChangeLog) can be seen below. ## v4.1 (2022-03-11) ### Changes * [flask] updated. [Alexandre Dulaunoy] * [flask] requirements for higher version of flask. [Alexandre Dulaunoy] * [v4.1] add Investigation with MISP Export + v4.1 update. [Terrtia] * [Telegram module] refactor module + fix str format. [Terrtia] ### Fix * [Investigation] edit misp event + add misp instance url. [Terrtia] * [Investigation] fix MISP Export + UI sidebar. [Terrtia] * [UI inestigations] add items link. [Terrtia] * [UI inestigations] add objects link. [Terrtia] * [telegram launcher] [Terrtia] * [items] abstract class. [Terrtia] * [Investigation] UI sidebar. [Terrtia] * [v4.1] fix ardb # tracking DB. [Terrtia] * [username] user icon. [Thirion Aurélien] * [Term tracker] fix item date. [Terrtia] * [Telegram module] fix launcher. [Terrtia] * [pybgpranking] package install. [Terrtia] * [popper install] rename popper repository. [Terrtia] https://github.com/floating-ui/floating-ui/discussions/1425 * [UI] remove update note. [Terrtia] * [trackers] fix get_all_items_sources. [Terrtia] * [crawler] fix is_splash_manager_connected #133. [Terrtia] ### Other * Merge branch 'master' of github.com:ail-project/ail-framework. [Terrtia] * Merge pull request #139 from gallypette/jabber-feeder. [Thirion Aurélien] add: [username] jabber support * Add: [username] jabber support. [Jean-Louis Huynen] * Merge branch 'master' of github.com:ail-project/ail-framework. [Terrtia] * Create SECURITY.md. [Alexandre Dulaunoy] 2022-03-14T15:39:04+00:00 rocksdb v7.0.2 rocksdb v7.0.2 2022-03-14T16:45:01+00:00 # Rocksdb Change Log ## 7.0.2 (03/12/2022) * Fixed a bug that DisableManualCompaction may assert when disable an unscheduled manual compaction. 2022-03-14T16:45:01+00:00 MISP v2.4.154 MISP v2.4.154 2022-03-18T11:52:08+00:00 MISP 2.4.154 released with a host of new features and fixes, including some new tools that help us navigate the current geo-political landscape when sharing information. # Sharing group blueprints Difficult times often call for radical measures, with the recent world events we've seen more and more communities rapidly reorganising as well as new large communities being established. Sharing information with only subsets of communities has become ever more important and whilst we've had the tools to facilitate this in MISP for a long time, rapidly managing different, often overlapping groups has been difficult. Sharing group blueprints allow us to programmatically define reusable blueprints for generating sharing groups, based on inheritance and various filters to automate the task of maintaining the groups. Sharing group blueprints accept JSON objects based on which they generate a sharing group each, where various filters can be set for the decision making. The syntax allows for boolean operators as well as the use of organisation metadata and existing sharing group inheritance. This can also be used to create derivative groups with certain members being excluded, for example the below would be such an example: ``` { "AND": { "OR": { "org_sector": "Financial", "sharing_group_id": 127 }, "NOT": { "org_nationality": [ "Russia", "Russian Federation", "Belarus", "Republic of Belarus" ] } } } ``` The above would generate a sharing group out of all organisations present in sharing group 127, any organisation that has "Financial" as its type, but excluding any of the specifically negated countries' orgnaisations. This system thrives on well maintained organisation lists, so make sure that you put in the extra effort of contextualising your organisations! Once a blueprint is created, you can review the organisations to be included and if you are satisfied, create the actual sharing group by clicking on (re)generate sharing group. ![sharing-group-blueprint](https://user-images.githubusercontent.com/3668672/158998299-52bfc259-ad7a-43a7-8287-a1f368cc9845.png) One of the advantages of this system is that the regeneration can be run at any time, for a single sharing group or for all, via the interface or the API. This means that creating a cron job that updates all sharing groups based on the rules regularly is trivial, ensuring that for example inherited organisations via updated child sharing groups are updated continuously. # Populate events using MISP JSON elements There's a new way to populate an individual, existing event: by uploading a JSON file containing MISP elements (such as attributes, objects, tags, galaxies, etc), one can now easily paste JSON blobs into a form that an be accessed by clicking on "Populate from..." and selecting "Populate using a JSON file containing MISP event content data". # Improvements to the OIDC authentication A host of improvements and fixes, including the switch to a new library, developed by Jakub Onderka. # Acknowledgement We would like to thank all the [contributors](https://www.misp-project.org/contributors), reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in [misp-objects](https://www.misp-project.org/objects.html), [misp-taxonomies](https://www.misp-project.org/taxonomies.html) and [misp-galaxy](https://www.misp-project.org/galaxy.html). As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. 2022-03-18T11:52:08+00:00 MISP v2.4.155 MISP v2.4.155 2022-03-18T12:41:30+00:00 This release is a rapid follow up to v2.4.154, addressing several rather annoying issues # Bugfixes - Various bugfixes to the sharing group blueprint system (especially to it being more restrictive than intended) - Updating the DB schema to avoid the diagnostics complaining - Fixed an issue with organisation meta fields defaulting to null rather than '' (causing the blueprint issue mentioned above) - Rework of the DB schema dumper - Fixes to the Kali Linux installer # Acknowledgement We would like to thank all the [contributors](https://www.misp-project.org/contributors), reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in [misp-objects](https://www.misp-project.org/objects.html), [misp-taxonomies](https://www.misp-project.org/taxonomies.html) and [misp-galaxy](https://www.misp-project.org/galaxy.html). As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. 2022-03-18T12:41:30+00:00 MISP v2.4.156 MISP v2.4.156 2022-03-18T16:22:37+00:00 We are pleased to announce the immediate availability of MISP v2.4.156 - a release bringing several new features and fixes two critical vulnerabilities. **We highly encourage everyone to update to this version as soon as possible**. # Protected mode - cryptographic signing of synchronisation With the current tensions, information assurance in many ways is becoming more and more important across the different MISP communities. Whilst foul play is often quickly discovered and leads to the ejection from a sharing community, leading to an inherent self-healing mechanism of the different networks, in some cases due to information's criticality, more active measures are needed. By design, MISP's sharing mechanisms rely on trust relationships between the different interconnected nodes in the various MISP networks. This means that in a mesh network of MISP nodes, information can travel via trusted synchronisation users, the information's veracity being ensured by the various site administrators of the different instances. In some cases this is not enough, especially when exchanging data that is meant to be adhered to blindly in a highly automated fashion. Vetted block lists for example affecting large constituencies and the automatic blocking of traffic for service providers for example. To support this use-case, MISP as of v2.4.156 has a new mechanism that allows event creators to attach a set of PGP instance signing keys to an event, which are used to sign the events on each hop of the synchronisation. This allows recipient MISPs to discard any updates coming from nodes that cannot produce a valid signature with one of the initial signing keys. ## An example Alice and Bob each have their own MISP instances, with Alice feeding Bob with critical information. Bob trusts this information immediately and blindly. Eвa, wanting to remove data points or diluting the information from Alice's stream, is also part of their broader network. Traditionally, Alice sharing an event to the network would propagate to both Bob's and Eвa's instance. Eвa could in this case abuse her administrative privileges to modify the event, perhaps injecting disinformation and removing valid data. By synchronising this back to Bob, Bob's instance would see an incoming synchronised edit, which in a mesh network could be legitimate and as such it would accept the change. Propagating it further back to Alice would be blocked by MISP's protection against remote modifications to data at origin. ![unprotected_sync_mode](https://user-images.githubusercontent.com/3668672/159035794-918f9c33-74dc-44e2-84db-34fdb1ba726a.png) With protected mode enabled, this situation changes drastically. Alice could add her own signing key as well as Bob's to the event, ensuring that the only parties able to relay modifications to the event would be Alice and Bob. When leaving Alice's instance, the event would get signed with Alice's signing key. Since the event contains both Alice's and Bob's key, any subsequent modifications from Alice would be accepted by Bob's instance. Incoming edits would be signed by Alice's key, meaning that Bob would validate the package with its locally stored public key of from the initial exchange. This means that Eвa modifying the event and attempting to share it with Bob would get rejected, as Eвa, lacking the private keys of Alice and Bob, can only sign it with her own key, which Bob's instance would immediately flag as suspicious and ultimately reject it. ![protected_sync_mode](https://user-images.githubusercontent.com/3668672/159036489-f2d457aa-cb23-42a8-b10b-6d9e9a02e7f9.png) ## Usage To get started with the feature, simply use the new protected mode field in the event view, you can convert any event into protected mode: ![protected1](https://user-images.githubusercontent.com/3668672/159038886-d82a516b-1281-4649-ac2d-ea456f7468ed.png) At which point you can start adding individual keys: ![protected2](https://user-images.githubusercontent.com/3668672/159039506-e896ca51-7a96-4f3f-a46b-718df9e0072c.png) Keep in mind that you can add multiple instance signing keys if you wish for your trusted partners (or your own instances, for example if you have an internal and a sharing MISP in the DMZ). ![protected3](https://user-images.githubusercontent.com/3668672/159039670-eb1e3b3d-8089-45c0-9511-aaf0ffc80b89.png) As a caveat, keep in mind that this mechanism limits the distribution of data inadvertently. Even if the distribution level would allow it, the synchronisation will be limited by who can sign the event for further propagation, so use this new functionality when the use-case really calls for it. A massive thank you to our good friend [Trey Darley](https://twitter.com/treyka) (@treyka) of Cert.be for the brainstorming session that lead to the implementation of this feature! # Context summary export A new export format was added that generates an HTML representation of a summary of all context information from a set of filtered data. One could for example use restsearch to generate all context from any event that is attributed to a threat actor. The resulting HTML will include the Mitre ATT&CK matrix of all leveraged techniques in the selected events as well as any other labelling and context. # Event warning system The new warning system warns users about potential improvements to an event they could be making, such as resolving tagging issues, improving the quality of the event, etc. The system comes with a plugable module system, easily build and deploy your own warning system. ![warning_system](https://user-images.githubusercontent.com/3668672/159040894-99d951e3-a076-40c7-9bbd-9ff619df2e5c.png) # Internal reworks @JakubOnderka continues his massive crusade against ugly spaghetti code, with a continuous stream of refactorings, this time massively improving the code-base of the synchronisation mechanism. # Pentest - Several security issues resolved We would like to thank Ianis BERNARD of NATO Cyber Security Centre. Based on the findings of their pentest we were able to resolve several security vulnerabilities and as such we highly encourage everyone to update to v2.4.156 ASAP. ## Security fixes resolved Four security vulnerabilities were fixed in this release. We strongly recommend everyone to install this version as soon as possible. - CVE-2022-27245 - [Potential SSRF attacks fixed](https://github.com/MISP/MISP/commit/8dcf414340c5ddedfebbc972601646d38e1d0717) on generateServerSettings(), the interface is now restricted to the cli interface only. - CVE-2022-27243 - [Potential LFI attack fixed](https://github.com/MISP/MISP/commit/8cc93687dcd68e1774b55a5c4e8125c0c8ddc288) via custom file setting. - CVE-2022-27246 - [Restrict SVG logo](https://github.com/MISP/MISP/commit/08a07a38ae81f3b55d81cfcd4501ac1eb1c9c4dc) upload for organisation by default and make it optional to limit potential risk of SVG with active payload. - CVE-2022-27244 - [Stored XSS in the user add/edit forms fixed](https://github.com/MISP/MISP/commit/61d4d3670593b78e4dab7a11eb620b7a372f30e6) in custom auth name with a potential malicious administrator. # LinOTP auth improvements Thanks to the lovely work submitted by @andurin, the LinOTP authentication subsystem now includes several improvements, amongst others the ability to conveniently manage and disable the subsystem directly via the system settings. Originally, the only way to disable the LinOTP authentication was to purge the related settings from the configuration files. In order not to break the expected functionality for users that already have LinOTP configured, the default behaviour for the new "LinOTP.enable" setting behaves a bit different from other similar settings: When no value was assigned by an administrator, the module is enabled by default if the LinOTP configuration keys exist in the configuration file. That means, if you've had it configured from before, by default it will be enabled. Other than that it will be disabled. Confirming the setting as either enabled or disabled by an administrator will override this behaviour with the selected setting. # A long list of other improvements We have received a massive list of pull requests for enhancements and fixes. Make sure you check out the [changelog](https://www.misp-project.org/Changelog.txt) for further details. # Acknowledgement We would like to thank all the [contributors](https://www.misp-project.org/contributors), reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in [misp-objects](https://www.misp-project.org/objects.html), [misp-taxonomies](https://www.misp-project.org/taxonomies.html) and [misp-galaxy](https://www.misp-project.org/galaxy.html). As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. 2022-03-18T16:22:37+00:00 PeekabooAV v2.1rc1 PeekabooAV v2.1rc1 2022-03-21T06:57:19+00:00 Install using `./setup.py`. Uninstallable and yanked from pypi.org due to dependency on our modified cortex4py github repo. See [CHANGELOG.md](https://github.com/scVENUS/PeekabooAV/blob/v2.1rc1/CHANGELOG.md) for changes since last release. 2022-03-21T06:57:19+00:00 syncthing v1.19.2-rc.2 syncthing v1.19.2-rc.2 2022-03-22T19:31:51+00:00 Enhancements: - #8180: Make error message upon reaching the free space quota more clear 2022-03-22T19:31:51+00:00 rocksdb v6.29.4 rocksdb v6.29.4 2022-03-23T01:24:02+00:00 ## 6.29.4 (03/22/2022) ### Bug Fixes * Fixed a bug caused by race among flush, incoming writes and taking snapshots. Queries to snapshots created with these race condition can return incorrect result, e.g. resurfacing deleted data. * Fixed a bug that DisableManualCompaction may assert when disable an unscheduled manual compaction. * Fixed a bug that `Iterator::Refresh()` reads stale keys after DeleteRange() performed. * Fixed a race condition when disable and re-enable manual compaction. * Fix a race condition when cancel manual compaction with `DisableManualCompaction`. Also DB close can cancel the manual compaction thread. * Fixed a data race on `versions_` between `DBImpl::ResumeImpl()` and threads waiting for recovery to complete (#9496) * Fixed a read-after-free bug in `DB::GetMergeOperands()`. * Fixed NUM_INDEX_AND_FILTER_BLOCKS_READ_PER_LEVEL, NUM_DATA_BLOCKS_READ_PER_LEVEL, and NUM_SST_READ_PER_LEVEL stats to be reported once per MultiGet batch per level. 2022-03-23T01:24:02+00:00 PeekabooAV v2.1rc2 PeekabooAV v2.1rc2 2022-03-23T08:40:59+00:00 Install using venv/bin/pip install peekabooav==2.1rc2. See [CHANGELOG.md](https://github.com/scVENUS/PeekabooAV/blob/v2.1rc2/CHANGELOG.md) for changes since last release. 2022-03-23T08:40:59+00:00 DC3-MWCP 3.6.0 DC3-MWCP 3.6.0 2022-03-23T23:35:22+00:00 2022-03-23T23:35:22+00:00 MISP v2.4.157 MISP v2.4.157 2022-03-25T14:21:20+00:00 We are pleased to announce the immediate availability of MISP v2.4.157, following a series of bug fixes as a quick follow up to 2.4.156. As a reminder, MISP v2.4.156 included several critical vulnerability fixes, as such, **we highly encourage everyone to update to this version as soon as possible**. It also brought several new important features that help communities ensure the veracity of their most critical shared data. # Fixes to the authkey handling Manage auth keys of your team as an org admin, until now this feature was broken and org admins had to log in as their automation / sync users in order to generate new keys. This is no longer the case, simply view the user you wish to create a new key for and do it directly from the interface or via the API. Keep in mind that org admins can only create keys for non administrator users. Thank you to @oivindoh for pointing this shortcoming out. # Fix to a breaking bug with event publishing Due to a bug introduced by a regression in 2.4.156, publishing events ended up not pushing events with sharing groups to remote instances. This is now resolved and for this in itself we already highly recommend updating to this version. Full instance pushes and pulls were not affected. Neither were events that didn't rely on sharing groups as their distribution model. Thank you to @treyka for finding the bug. # New setting introduced to disable event lock checks Sometimes the addition of certain features, whilst having good intentions, ends up being more annoying that useful. In these cases, unless it's something absolutely hindering, we still do not want to modify the default behaviour of MISP over night. Such is the case with the event lock checks, which provide warnings on the event view that another user is also editing the event, a simple warning to users that their event's state may be outdated. This functionality is rather verbose when it comes to logging, gets in the way of debugging and can cause session persistence issues in certain cases. As such we've introduced a new setting to disable the functionality and unless you or your community are especially attached to it, we recommend heading over to the server settings and disabling it via the `MISP.disable_event_locks` setting. Thanks to @github-germ and @packet-rat for pointing the annoying nature of this feature out. # Acknowledgement We would like to thank all the [contributors](https://www.misp-project.org/contributors), reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in [misp-objects](https://www.misp-project.org/objects.html), [misp-taxonomies](https://www.misp-project.org/taxonomies.html) and [misp-galaxy](https://www.misp-project.org/galaxy.html). As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. 2022-03-25T14:21:20+00:00 rocksdb v7.0.3 rocksdb v7.0.3 2022-03-25T17:00:11+00:00 ### Bug Fixes * Fixed a major performance bug in which Bloom filters generated by pre-7.0 releases are not read by early 7.0.x releases (and vice-versa) due to changes to FilterPolicy::Name() in #9590. This can severely impact read performance and read I/O on upgrade or downgrade with existing DB, but not data correctness. * Fixed a bug that `Iterator::Refresh()` reads stale keys after DeleteRange() performed. ### Public API changes * Added pure virtual FilterPolicy::CompatibilityName(), which is needed for fixing major performance bug involving FilterPolicy naming in SST metadata without affecting Customizable aspect of FilterPolicy. For source code, this change only affects those with their own custom or wrapper FilterPolicy classes, but does break compiled library binary compatibility in a patch release. 2022-03-25T17:00:11+00:00 DC3-MWCP 3.6.1 DC3-MWCP 3.6.1 2022-03-28T16:55:28+00:00 2022-03-28T16:55:28+00:00 rocksdb v6.29.5 rocksdb v6.29.5 2022-03-29T20:30:03+00:00 ## 6.29.5 (03/29/2022) ### Bug Fixes * Fixed a race condition for `alive_log_files_` in non-two-write-queues mode. The race is between the write_thread_ in WriteToWAL() and another thread executing `FindObsoleteFiles()`. The race condition will be caught if `__glibcxx_requires_nonempty` is enabled. * Fixed a race condition when mmaping a WritableFile on POSIX. * Fixed a race condition when 2PC is disabled and WAL tracking in the MANIFEST is enabled. The race condition is between two background flush threads trying to install flush results, causing a WAL deletion not tracked in the MANIFEST. A future DB open may fail. * Fixed a heap use-after-free race with DropColumnFamily. * Fixed a bug that `rocksdb.read.block.compaction.micros` cannot track compaction stats (#9722). 2022-03-29T20:30:03+00:00 rocksdb v7.0.4 rocksdb v7.0.4 2022-03-29T20:31:07+00:00 ## 7.0.4 (03/29/2022) ### Bug Fixes * Fixed a race condition when disable and re-enable manual compaction. * Fixed a race condition for `alive_log_files_` in non-two-write-queues mode. The race is between the write_thread_ in WriteToWAL() and another thread executing `FindObsoleteFiles()`. The race condition will be caught if `__glibcxx_requires_nonempty` is enabled. * Fixed a race condition when mmaping a WritableFile on POSIX. * Fixed a race condition when 2PC is disabled and WAL tracking in the MANIFEST is enabled. The race condition is between two background flush threads trying to install flush results, causing a WAL deletion not tracked in the MANIFEST. A future DB open may fail. * Fixed a heap use-after-free race with DropColumnFamily. * Fixed a bug that `rocksdb.read.block.compaction.micros` cannot track compaction stats (#9722). 2022-03-29T20:31:07+00:00 Lookyloo v1.11.0 Lookyloo v1.11.0 2022-03-31T11:17:11+00:00 # New Feature * Trigger multiple captures at once from web interface ![Screenshot_20220331_131600](https://user-images.githubusercontent.com/248875/161043117-d56ac8ce-aaeb-4cd6-bb55-87cb7c5e93e1.png) # Fixes * Improve MISP event publishing (make it asynchronous) * Improve legend with titles on hoover * Fix caches in modules * Improve stats page * Normalize buttons color * Improve rendering of capture page # Changes * Updates all web and python dependencies * Use bootstrap 5 2022-03-31T11:17:11+00:00 maltrail 0.44 maltrail 0.44 2022-03-31T22:11:06+00:00 Start-of-month release 2022-03-31T22:11:06+00:00 DC3-MWCP 3.6.2 DC3-MWCP 3.6.2 2022-04-04T20:01:55+00:00 2022-04-04T20:01:55+00:00 syncthing v1.19.2 syncthing v1.19.2 2022-04-05T04:47:28+00:00 Enhancements: - #8180: Make error message upon reaching the free space quota more clear 2022-04-05T04:47:28+00:00 osquery 5.2.3 osquery 5.2.3 2022-04-05T22:05:20+00:00 Full Commits: https://github.com/osquery/osquery/compare/5.2.2...5.2.3 2022-04-05T22:05:20+00:00 osv v0.0.14 osv v0.0.14 2022-04-06T03:09:00+00:00 2022-04-06T03:09:00+00:00 TheHive 4.1.19 TheHive 4.1.19 2022-04-07T14:43:23+00:00 ## [4.1.19](https://github.com/TheHive-Project/TheHive/milestone/89) (2022-04-07) **Implemented enhancements:** - Migrate hive3 to 4, migrate SSL error. Requesting SSL bypass option [\#2356](https://github.com/TheHive-Project/TheHive/issues/2356) - [Enhancement] Improve logging [\#2371](https://github.com/TheHive-Project/TheHive/issues/2371) **Fixed bugs:** - [Bug] Regression starting with 4.1.17 in the migration tool with certificate validation [\#2342](https://github.com/TheHive-Project/TheHive/issues/2342) - [Bug] Deadlock when the index backend changed [\#2351](https://github.com/TheHive-Project/TheHive/issues/2351) - [Bug][Security] TheHive4 libraries vulnerabilities [\#2362](https://github.com/TheHive-Project/TheHive/issues/2362) - [Bug] The Link given in /etc/thehive/application.conf says error. [\#2364](https://github.com/TheHive-Project/TheHive/issues/2364) - [Bug] An unshared case is still visible [\#2366](https://github.com/TheHive-Project/TheHive/issues/2366) - [Bug] Prevent custom field creation if it already exists [\#2367](https://github.com/TheHive-Project/TheHive/issues/2367) - [Bug] An user may exist without being member of any organisation [\#2368](https://github.com/TheHive-Project/TheHive/issues/2368) - [Bug] Tags can't be updated [\#2369](https://github.com/TheHive-Project/TheHive/issues/2369) - [Bug] Dashboarding on weeks provide incorrect results on specific dates [\#2370](https://github.com/TheHive-Project/TheHive/issues/2370) 2022-04-07T14:43:23+00:00 reko version-0.11.1 reko version-0.11.1 2022-04-07T22:17:52+00:00 This maintenance release provides minor enhancements and bugfixes, including: * More ARM32 rewriters. * More uses of generic `IntrinsicProcedures`. * Replace recursive `SccFinder` with an non-recursive implementation. * Initial support for Terse Executable format. * Don't try tracing into nonexecutable code. * Ctrl+0 resets the zoom level of the Graph Viewer * Many more PowerPC instructions supported * C parser issues reported by @smx-smx * Make SSA analysis use bit-accurate analysis for stack variables. * Multithreaded robustness. It also has some refactorings and new classes, setting the stage for a future refactoring of the Scanner: * New `RtlSwitch` subclass of `RtlInstruction` * Support for platform-specific patterns for procedure entries. 2022-04-07T22:17:52+00:00 PeekabooAV v2.1rc3 PeekabooAV v2.1rc3 2022-04-13T10:41:50+00:00 Install using venv/bin/pip install peekabooav==2.1rc3. See [CHANGELOG.md](https://github.com/scVENUS/PeekabooAV/blob/v2.1rc3/CHANGELOG.md) for changes since last release. 2022-04-13T10:41:50+00:00 OpenTAXII 0.2.4 OpenTAXII 0.2.4 2022-04-13T11:19:28+00:00 Changelog ========= 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-04-13T11:19:28+00:00 OpenTAXII 0.3.0a4 OpenTAXII 0.3.0a4 2022-04-13T12:20:59+00:00 Changelog ========= 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-04-13T12:20:59+00:00 OpenTAXII 0.3.0 OpenTAXII 0.3.0 2022-04-13T13:25:40+00:00 Changelog ========= 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-04-13T13:25:40+00:00 caddy v2.5.0-rc.1 caddy v2.5.0-rc.1 2022-04-13T18:56:51+00:00 Caddy 2.5 introduces new features you'll love as well as a huge number of bug fixes and enhancements. Thank you to everyone who contributed. **:warning: This is a release candidate version. We think it's stable enough to use in production, but we want to be extra sure, so please try it out! It needs to be used and tested for regressions. Let us know if there are any issues.** Documentation on the website has mostly been updated (JSON docs will be last). Feel free to ask on the [forum](https://caddy.community) if you have any questions or feedback! ## Highlights - **Reverse proxy:** [:sparkles: _Dynamic upstreams_](https://github.com/caddyserver/caddy/pull/4470), which is the ability to get the list of upstreams at every request (more specifically, every iteration in the proxy loop of every request) rather than just once at config-load time. Dynamic upstream modules can be plugged in to provide Caddy with the latest list of backends in real-time. Two standard modules have been implemented which can get upstreams from SRV and A/AAAA record lookups. - :warning: This deprecates the `lookup_srv` JSON field for upstreams (and `srv+` scheme prefix in the Caddyfile), which will be removed in the future. - **Automatic HTTPS:** Caddy will automatically try to get relevant certificates from the local [Tailscale](https://tailscale.com) instance (if running with permission to access the Tailscale socket). This makes services running on a Tailscale network [automatically available](https://github.com/caddyserver/caddy/pull/4541) over trusted HTTPS with Caddy. - **Tracing:** New [OpenTelemetry](https://opentelemetry.io/) integration with the [`tracing` handler module and associated `tracing` directive](https://github.com/caddyserver/caddy/pull/4361). - **Reverse proxy:** When using the response handlers, a new handler `copy_response` is available to copy the proxy's response back to the client, and `copy_response_headers` may be used to selectively copy header values from the proxy's response. - **API:** Added new endpoints `/pki/ca/<id>` and `/pki/ca/<id>/certificates` for getting information about Caddy's managed CAs, including the chain of root and intermediate certificates. ## Notable - **Reverse proxy:** The `X-Forwarded-Host` header will now be automatically set, along with `X-Forwarded-For` and `X-Forwarded-Proto`. - :warning: **Reverse proxy:** Incoming `X-Forwarded-*` headers will no longer be automatically trusted, to prevent spoofing. Now, `trusted_proxies` must be configured to specify a list of downstream proxies which are trusted to have sent good values. You only need to configure trusted proxies if Caddy is not the first server being connected to. For example, if you have Cloudflare in front of Caddy, then you should configure this with Cloudflare's [list of IP ranges](https://www.cloudflare.com/en-ca/ips/). - **Automatic HTTPS:** Revoked certificates will be automatically [replaced more reliably](https://github.com/caddyserver/certmagic/pull/166). - **Automatic HTTPS:** [Can now get certificates from _Managers_.](https://github.com/caddyserver/caddy/pull/4541) As opposed to _Issuers_ (such as the default ACME issuers) which give Caddy certificates to manage from a CSR, Managers give Caddy certificates to serve (rather than manage) during TLS handshakes. - **Automatic HTTPS:** A DNS challenge [domain override](https://github.com/caddyserver/caddy/issues/4071) can be configured to delegate the solving of the challenge to a different domain. - **Reverse proxy:** The default dial timeout for the HTTP transport [has been adjusted](https://github.com/caddyserver/caddy/pull/4436) down to `3s` (was `10s`), which should allow for more easily configuring load balancing retries. - **Logging:** HTTP access logs will now [render empty values](https://github.com/caddyserver/caddy/commit/7d5047c1f190421528695e1cc3a4ad71c97eb022) for often-sensitive HTTP headers such as Cookie, Authorization, and Proxy-Authorization. Logging such credentials is now [opt-in](https://github.com/caddyserver/caddy/commit/5bf0adad8748e96e10529d5fc5777afc9236a7b5) with the `log_credentials` global option in the Caddyfile, or the server's `logs > should_log_credentials` field in JSON. - **Logging:** Logs can now be filtered by [query string parameters](https://github.com/caddyserver/caddy/commit/bcac2beee7e419f8cdab2ed16f388d1af282a46b), [cookie values](https://github.com/caddyserver/caddy/commit/8887adb027982e844965b4707b8595cee5845d54), and [regular expressions](https://github.com/caddyserver/caddy/commit/789efa5deef53071b57479d37e4022bf372c4eef); and log values can be [hashed](https://github.com/caddyserver/caddy/commit/a1b417c832b4ab3dab9eaa9690e1d07672a949b8). These features are useful for redacting sensitive information. - **Logging:** Errors during request handling [will now be logged at `DEBUG` level](https://github.com/caddyserver/caddy/pull/4429) if the error was [handled via `errors` routes](https://github.com/caddyserver/caddy/pull/4584) (`handle_errors` in Caddyfile). - :warning: **Logging:** Removed the [deprecated](https://github.com/caddyserver/caddy/issues/4148) `common_log` field from HTTP access logs, and the `single_field` encoder. If you relied on this, you may use the [transform encoder plugin](https://github.com/caddyserver/transform-encoder) to encode logs in Common Log format. - :warning: **Logging:** The `remote_addr` field [has been replaced](https://github.com/caddyserver/caddy/commit/f55b123d63132e290789bcd07077375c76b6e1dd) by `remote_ip` and `remote_port` fields in HTTP access logs, which split up the two parts of the remote address. This improves ease of use for some tooling which only expect an IP address, without a port. - **HTTP server:** The [`vars` matcher](https://github.com/caddyserver/caddy/commit/ecac03cdcb6cceae743aac16faca7f32e5da1607) can now match on multiple possible values. - **HTTP server:** Requests [can now be assigned](https://github.com/caddyserver/caddy/commit/180ae0cc4843ecc3c7ddcb6e978ebfd474ed07f9) a random and unique UUID from the new `{http.request.uuid}` placeholder. - **HTTP server:** [New `http_redirect` listener wrapper](https://github.com/caddyserver/caddy/pull/4585) which can be used to redirect HTTP requests that come in on a server listening for HTTPS requests to be redirected to `https://`. - **Caddyfile:** [New `default_bind` global option](https://github.com/caddyserver/caddy/pull/4531) lets you specify the default interface all sockets should bind to. - **Caddyfile:** [New `pki` global option](https://github.com/caddyserver/caddy/pull/4450) lets you configure the properties of the internal CAs managed by Caddy. - **Caddyfile:** [New `method` directive](https://github.com/caddyserver/caddy/pull/4528) allows rewriting the request method via Caddyfile. - :warning: **Caddyfile:** The `reverse_proxy` directive's `handle_response` subdirective has had its status replacement functionality [moved to a new `replace_status`](https://github.com/caddyserver/caddy/pull/4300) subdirective. This makes sure that the functionality of `handle_response` is not overloaded, and usage is clearer. - :warning: **Admin:** [Renamed](https://github.com/caddyserver/caddy/commit/bc447e307f195b80eeec0f6157e0d8e641af9155) experimental property `load_interval` :arrow_right: `load_delay` for clarification, and improved dynamic config loading. --- :shield: Thanks to [David Leadbeater](https://github.com/dgl) for reporting a security vulnerability related to HTTP methods and metrics cardinality, which was fixed in this release. ## New Contributors * @adamburgess made their first contribution in https://github.com/caddyserver/caddy/pull/4460 * @12f23eddde made their first contribution in https://github.com/caddyserver/caddy/pull/4444 * @rayjlinden made their first contribution in https://github.com/caddyserver/caddy/pull/4023 * @GallopingKylin made their first contribution in https://github.com/caddyserver/caddy/pull/4522 * @ForestJohnson made their first contribution in https://github.com/caddyserver/caddy/pull/4534 * @VojtechVitek made their first contribution in https://github.com/caddyserver/caddy/pull/4535 * @Ikke made their first contribution in https://github.com/caddyserver/caddy/pull/4544 * @YourTechBud made their first contribution in https://github.com/caddyserver/caddy/pull/4603 * @BitWuehler made their first contribution in https://github.com/caddyserver/caddy/pull/4597 * @ttys3 made their first contribution in https://github.com/caddyserver/caddy/pull/4572 * @crccw made their first contribution in https://github.com/caddyserver/caddy/pull/4596 * @andriikushch made their first contribution in https://github.com/caddyserver/caddy/pull/4361 * @renbou made their first contribution in https://github.com/caddyserver/caddy/pull/4654 * @cuishuang made their first contribution in https://github.com/caddyserver/caddy/pull/4702 ## Changelog * 2e46c2ac admin, reverseproxy: Stop timers if canceled to avoid goroutine leak (#4482) * 40b54434 admin: Enforce and refactor origin checking * b4bfa29b admin: Require identity for remote (fix #4478) * 32aad909 admin: Write proper status on invalid requests (#4569) (fix #4561) * ff137d17 caddyconfig: Support placeholders in HTTP loader * b47af6ef caddyfile: Copy input before parsing (fix #4422) * e90d7517 caddyfile: impove fmt warning message (#4444) * 5e5af50e caddyfile: make renew_interval option configurable (#4451) * ddbb234d caddyhttp: Always log handled errors at debug level (#4584) * 6b385a36 caddyhttp: Don't attempt to manage Tailscale certs * ecac03cd caddyhttp: Enhance vars matcher (#4433) * 6e6ce2be caddyhttp: Fix HTTP->HTTPS redir not preferring HTTPS port if ambiguous (#4530) * 3fe2c73d caddyhttp: Fix `MatchPath` sanitizing (#4499) * 44e5e9e4 caddyhttp: Fix test when /tmp/etc already exists (#4544) * 2bb8550a caddyhttp: Honor wildcard hosts in log SkipHosts (#4606) * 180ae0cc caddyhttp: Implement http.request.uuid placeholder (#4285) * 7d5047c1 caddyhttp: Log empty value for typical password headers * eead3373 caddyhttp: Log non-500 handler errors at debug level (#4429) * 5bf0adad caddyhttp: Make logging of credential headers opt-in (#4438) * 186fdba9 caddyhttp: Move HTTP redirect listener to an optional module (#4585) * 80d7a356 caddyhttp: Redirect HTTP requests on the HTTPS port to https:// (#4313) * bf380d00 caddyhttp: Reject absurd methods (#4538) * 850e1605 caddyhttp: Return HTTP 421 for mismatched Host header (#4023) * f55b123d caddyhttp: Split up logged remote address into IP and port (#4403) * ac14b64e caddyhttp: Support zone identifiers in remote_ip matcher (#4597) * a1c41210 caddypki: Minor tweak, don't use context pointer * 78e381b2 caddypki: Refactor /pki/ admin endpoints * c634bbe9 caddypki: Return error if no PEM data found * 9b7cdfa2 caddypki: Try to fix lint warnings * a79b4055 caddytls: Add internal Caddyfile `lifetime`, `sign_with_root` opts (#4513) * 66de438a caddytls: Fix `MatchRemoteIP` provisoning with multiple CIDR ranges (#4522) * 57a708d1 caddytls: Support external certificate Managers (like Tailscale) (#4541) * d9b1d463 caddytls: dns_challenge_override_domain for challenge delegation (#4596) * 1a7a78a1 cmd: Print error if fmt overwrite fails (fix #4524) * bc447e30 core: Config LoadInterval -> LoadDelay for clarity * 7ea5b2a8 core: Config load interval only reloads if changed (#4603) * 7f364c77 core: Load config at interval instead of just once * a72acd21 core: Retry dynamic config load if config unchanged * ceef70db core: Retry dynamic config load if error or no-op (#4603) * acbee947 core: Revert 7f364c7; simplify dynamic config load * 64a3218f core: Simplify shared listeners, fix deadline bug * 8e5aafa5 fastcgi: Fix a TODO, prevent zap using reflection for logging env (#4437) * c8f2834b fastcgi: Protect against requests with null bytes in the path (#4614) * de490c7c fastcgi: Set SERVER_PORT to 80 or 443 depending on scheme (#4572) * 09ba9e99 fileserver: Add `pass_thru` Caddyfile option (#4613) * 15c95e9d fileserver: Canonical redir when whole path is stripped (#4549) * c8b5a816 fileserver: Fix handling of symlink sizes in directory listings (#4415) * e81369e2 fileserver: Move default browse template into a separate file (#4417) * 1e10f6f7 fileserver: browse: do not encode the paths in breadcrumbs and page title (#4410) * 78b5356f fileserver: do not double-escape paths (#4447) * 0de51593 go.mod: Revert version bump of CEL (#4587) * 6f9b6ad7 go.mod: Update smallstep/certificates, no longer need replace (#4475) * 4906b935 go.mod: Update smallstep/truststore, fix build on FreeBSD (#4473) * c1331534 go.mod: Update to latest smallstep/truststore, support FreeBSD (#4453) * ff74a0aa go.mod: Upgrade dependencies * e9dde230 headers: Fix `+` in Caddyfile to properly append rather than set (#4506) * 1b7ff5d7 httpcaddyfile: Add `default_bind` global option (#4531) * 5a071568 httpcaddyfile: Add pki app `root` and `intermediate` cert/key config (#4514) * 26d633ba httpcaddyfile: Disabling OCSP stapling for both managed and unmanaged (#4589) * 93a7a45e httpcaddyfile: Fix incorrect handling of IPv6 bind addresses (#4532) * 81ee34e9 httpcaddyfile: Fix sorting edgecase for nested `handle_path` (#4477) * 4b9849c7 httpcaddyfile: Support configuring `pki` app names via global options (#4450) * 5bd96a6a httpcaddyfile: Support explicitly turning off `strict_sni_host` (#4592) * c921e082 logging: Add `roll_local_time` Caddyfile option (#4583) * 0eb0b60f logging: Remove common_log field and single_field encoder (#4149) (#4282) * 249adc1c logging: Support turning off roll compression via Caddyfile (#4505) * 8887adb0 logging: add a filter for cookies (#4425) * bcac2bee logging: add a filter for query parameters (#4424) * 789efa5d logging: add a regexp filter (#4426) * a1b417c8 logging: add support for hashing data (#4434) * eb891d46 metrics: Enforce smaller set of method labels * c04d24ca pki: Avoid provisioning the `local` CA when not necessary (#4463) * bbad6931 pki: Implement API endpoints for certs and `caddy trust` (#4443) * 9ee68c1b reverseproxy: Adjust defaults, document defaults (#4436) * 7557d1d9 reverseproxy: Avoid returning a `nil` error during GetClientCertificate (#4550) * ab045592 reverseproxy: Dynamic upstreams (with SRV and A/AAAA support) (#4470) * 5333c352 reverseproxy: Fix incorrect `health_headers` Caddyfile parsing (#4485) * c50094fc reverseproxy: Implement trusted proxies for `X-Forwarded-*` headers (#4507) * f5e10494 reverseproxy: Make shallow-ish clone of the request (#4551) * 87a1f228 reverseproxy: Move status replacement intercept to `replace_status` (#4300) * d058dee1 reverseproxy: Refactor dial address parsing, augment command parsing (#4616) * c7d6c4cb reverseproxy: copy_response and copy_response_headers for handle_response routes (#4391) * bcb7a19c rewrite: Add `method` Caddyfile directive (#4528) * 1feb6595 rewrite: Fix a double-encode issue when using the `{uri}` placeholder (#4516) * 6cadb60f templates: Document .OriginalReq * 1d0425b2 templates: Elaborate on what's supported by the markdown function (#4564) * a6199cf8 templates: Fix docs for .Args * ec14ccdd templates: fix inconsistent nested includes (#4452) * d0b608af tracing: New OpenTelemetry module (#4361) * 134b8056 caddyfile: Prevent bad block opening tokens (#4655) * c5fffb4a caddyfile: Support for raw token values; improve `map`, `expression` (#4643) * b82e22b4 caddyhttp: retain all values of vars matcher when specified multiple times (#4629) * bc15b4b0 caddypki: Load intermediate for signing on-the-fly (#4669) * 6512832f cmd: Add `--diff` option for `caddy fmt` (#4695) * 30b6d1f4 cmd: Enhance .env (dotenv) file parsing * 22d8edb9 cmd: Fix defaulting admin address if empty in config, fixes `reload` (#4674) * c2327161 cmd: Set Origin header properly on API requests * d06d0e79 go.mod: Upgrade CertMagic to v0.16.0 * bf6a1b75 go.mod: Upgrade some dependencies * 79cbe7bf httpcaddyfile: Add 'vars' directive * a58f240d httpcaddyfile: Fix #4640 (auto-HTTPS edgecase) (#4661) * 7d229665 logging: Caddyfile support for `duration_format` (#4684) * 55b4c12e map: Evaluate placeholders in output vals (#4650) * 93c99f67 map: Support numeric and bool types with Caddyfile * 3d616e8c requestbody: Return HTTP 413 (fix #4558) * 3e3bb002 reverseproxy: Add `_ms` placeholders for proxy durations (#4666) * 2196c92c reverseproxy: Don't clear name in SRV upstreams * dc4d1473 reverseproxy: Expand SRV/A addrs for cache key * b8dbecb8 reverseproxy: Include port in A upstreams cache * e4ce40f8 reverseproxy: Sync up `handleUpgradeResponse` with stdlib (#4664) * afca2421 staticfiles: Expand placeholder for index files (#4679) * 00234c8a templates: Switch to `BurntSushi/toml` (#4700) **Full Changelog**: https://github.com/caddyserver/caddy/compare/v2.4.6...v2.5.0-rc.1 2022-04-13T18:56:51+00:00 rocksdb v7.1.1 rocksdb v7.1.1 2022-04-13T21:20:40+00:00 ## 7.1.1 (04/07/2022) ### Bug Fixes * Fix segfault in FilePrefetchBuffer with async_io as it doesn't wait for pending jobs to complete on destruction. ## 7.1.0 (03/23/2022) ### New Features * Allow WriteBatchWithIndex to index a WriteBatch that includes keys with user-defined timestamps. The index itself does not have timestamp. * Add support for user-defined timestamps to write-committed transaction without API change. The `TransactionDB` layer APIs do not allow timestamps because we require that all user-defined-timestamps-aware operations go through the `Transaction` APIs. * Added BlobDB options to `ldb` * `BlockBasedTableOptions::detect_filter_construct_corruption` can now be dynamically configured using `DB::SetOptions`. * Automatically recover from retryable read IO errors during backgorund flush/compaction. * Experimental support for preserving file Temperatures through backup and restore, and for updating DB metadata for outside changes to file Temperature (`UpdateManifestForFilesState` or `ldb update_manifest --update_temperatures`). * Experimental support for async_io in ReadOptions which is used by FilePrefetchBuffer to prefetch some of the data asynchronously, if reads are sequential and auto readahead is enabled by rocksdb internally. ### Bug Fixes * Fixed a major performance bug in which Bloom filters generated by pre-7.0 releases are not read by early 7.0.x releases (and vice-versa) due to changes to FilterPolicy::Name() in #9590. This can severely impact read performance and read I/O on upgrade or downgrade with existing DB, but not data correctness. * Fixed a data race on `versions_` between `DBImpl::ResumeImpl()` and threads waiting for recovery to complete (#9496) * Fixed a bug caused by race among flush, incoming writes and taking snapshots. Queries to snapshots created with these race condition can return incorrect result, e.g. resurfacing deleted data. * Fixed a bug that DB flush uses `options.compression` even `options.compression_per_level` is set. * Fixed a bug that DisableManualCompaction may assert when disable an unscheduled manual compaction. * Fix a race condition when cancel manual compaction with `DisableManualCompaction`. Also DB close can cancel the manual compaction thread. * Fixed a potential timer crash when open close DB concurrently. * Fixed a race condition for `alive_log_files_` in non-two-write-queues mode. The race is between the write_thread_ in WriteToWAL() and another thread executing `FindObsoleteFiles()`. The race condition will be caught if `__glibcxx_requires_nonempty` is enabled. * Fixed a bug that `Iterator::Refresh()` reads stale keys after DeleteRange() performed. * Fixed a race condition when disable and re-enable manual compaction. * Fixed automatic error recovery failure in atomic flush. * Fixed a race condition when mmaping a WritableFile on POSIX. ### Public API changes * Added pure virtual FilterPolicy::CompatibilityName(), which is needed for fixing major performance bug involving FilterPolicy naming in SST metadata without affecting Customizable aspect of FilterPolicy. This change only affects those with their own custom or wrapper FilterPolicy classes. * `options.compression_per_level` is dynamically changeable with `SetOptions()`. * Added `WriteOptions::rate_limiter_priority`. When set to something other than `Env::IO_TOTAL`, the internal rate limiter (`DBOptions::rate_limiter`) will be charged at the specified priority for writes associated with the API to which the `WriteOptions` was provided. Currently the support covers automatic WAL flushes, which happen during live updates (`Put()`, `Write()`, `Delete()`, etc.) when `WriteOptions::disableWAL == false` and `DBOptions::manual_wal_flush == false`. * Add DB::OpenAndTrimHistory API. This API will open DB and trim data to the timestamp specified by trim_ts (The data with timestamp larger than specified trim bound will be removed). This API should only be used at a timestamp-enabled column families recovery. If the column family doesn't have timestamp enabled, this API won't trim any data on that column family. This API is not compatible with avoid_flush_during_recovery option. * Remove BlockBasedTableOptions.hash_index_allow_collision which already takes no effect. 2022-04-13T21:20:40+00:00 syncthing v1.20.0-rc.1 syncthing v1.20.0-rc.1 2022-04-15T04:50:34+00:00 Bugfixes: - #7855: Watching for changes using too many system resources on BSDs (kqueue) - #8020: Dangling symlink prevents filesystem watcher on Solaris - #8232: Missing include file endlessly spams log - #8258: The latest version does not support go1.18 compilation!! Enhancements: - #8202: Indicate folders / devices where the remote end has not accepted sharing. Other issues: - #8279: Simplify tests using `T.TempDir` 2022-04-15T04:50:34+00:00 rocksdb v7.1.2 rocksdb v7.1.2 2022-04-20T01:48:15+00:00 ## 7.1.2 (04/19/2022) ### Bug Fixes * Fixed bug which caused rocksdb failure in the situation when rocksdb was accessible using UNC path * Fixed a race condition when 2PC is disabled and WAL tracking in the MANIFEST is enabled. The race condition is between two background flush threads trying to install flush results, causing a WAL deletion not tracked in the MANIFEST. A future DB open may fail. * Fixed a heap use-after-free race with DropColumnFamily. * Fixed a bug that `rocksdb.read.block.compaction.micros` cannot track compaction stats (#9722). * Fixed `file_type`, `relative_filename` and `directory` fields returned by `GetLiveFilesMetaData()`, which were added in inheriting from `FileStorageInfo`. * Fixed a bug affecting `track_and_verify_wals_in_manifest`. Without the fix, application may see "open error: Corruption: Missing WAL with log number" while trying to open the db. The corruption is a false alarm but prevents DB open (#9766). 2022-04-20T01:48:15+00:00 MISP v2.4.158 MISP v2.4.158 2022-04-20T07:43:37+00:00 We are pleased to announce the immediate availability of MISP v2.4.158. This release includes a series of security fixes and as such **we highly encourage everyone to update to this version as soon as possible**. Thanks to Dawid Czarnecki of Zigrin Security for the in-depth penetration test and its findings and thanks to the Luxembourg Army for financing the penetration test. This is the follow up to the Cerebrate penetration test also conducted by Zigrin Security on behalf of the Luxembourg Army, as described [here](https://www.cerebrate-project.org/2022/01/27/Cerebate-version-1.4-released.html). # Security fixes Several security issues have been resolved, head over to [the security page](https://www.misp-project.org/security/) for a detailed break-down of the advisories including the associated CVEs. Whilst most of the vulnerabilities listed are mitigated by requiring compromised high privilege accounts, we nevertheless advise all users to update their instances as soon as possible. - Phar deserialisation - [Global fix](https://github.com/MISP/MISP/commit/0108f1bde2117ac5c1e28d124128f60c8bb09a8e) - [Individual additional mitigations](https://github.com/MISP/MISP/commit/93821c0de6a7dd32262ce62212773f43136ca66e) - [XSS in LinOTP login](https://github.com/MISP/MISP/commit/9623de2f5cca011afc581d55cfa5ce87682894fd) - [XSS in Galaxy clusters](https://github.com/MISP/MISP/commit/107e271d78c255d658ce998285fe6f6c4f291b41) - [XSS in organisation fetchSGOrgRow](https://github.com/MISP/MISP/commit/ce6bc88e330f5ef50666b149d86c0d94f545f24e) - [XSS in Event graph via tags](https://github.com/MISP/MISP/commit/bb3b7a7e91862742cae228c43b3091bad476dcc0) - [XSS in Cerebrate view](https://github.com/MISP/MISP/commit/60c85b80e3ab05c3ef015bca5630e95eddbb1436) - [Password confirmation bypass](https://github.com/MISP/MISP/commit/01120163a6b4d905029d416e7305575df31df8af) ## Announcement of a silent fix of phar deserialisation RCE in a previous release (v2.4.156) As of the previous security release (v2.4.156), based on the pentest conducted by Ianis BERNARD of the NATO Cyber Security Centre, a high criticality vulnerability was also identiefied. We have opted for a silent fix to the critical vulnerability whilst upgrading the announced criticality of the other security fixes included in the release. This is an extreme measure that we take whenever we want to ensure that the community is both aware that they do need to update as soon as possible whilst not drawing attention to the actual critical vulnerability. If you have followed our guidance over the past month to update you are already safe - if you are running a MISP instance below 2.4.156 **we highly encourage you to update to the latest version as soon as possible**. - [Phar deserialisation silent fix](https://github.com/MISP/MISP/commit/8eff854fce1fea1521f33fffc2440df5b7e5c410) # Custom email templates Added the ability to override some of the standard e-mail templates with custom ones, just drop the templates mirroring the naming convention of the existing ones in `/var/www/MISP/app/View/Email/text` and `/var/www/MISP/app/View/Email/html` into `/var/www/MISP/app/View/Email/text/Custom/` and `/var/www/MISP/app/View/Email/html/Custom/`. Currently supported templates: alert, password_reset. # RestSearch improvements Fixing a baffling oversight on our side, thanks to Tom King we can now search by sharing groups besides just distribution levels. # A long list of refactors and bugfixes Massive thanks to Jakub Onderka for the continuous refactoring, simplifying and cleaning up of the code-base. For a full list of all the improvements that are part of this herculean effort, refer to the [changelog](https://www.misp-project.org/Changelog.txt) # Acknowledgement We would like to thank all the [contributors](https://www.misp-project.org/contributors), reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in [misp-objects](https://www.misp-project.org/objects.html), [misp-taxonomies](https://www.misp-project.org/taxonomies.html) and [misp-galaxy](https://www.misp-project.org/galaxy.html). As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. 2022-04-20T07:43:37+00:00 syncthing v1.20.0-rc.2 syncthing v1.20.0-rc.2 2022-04-21T13:57:00+00:00 Bugfixes: - #7855: Watching for changes using too many system resources on BSDs (kqueue) - #8020: Dangling symlink prevents filesystem watcher on Solaris - #8232: Missing include file endlessly spams log - #8258: The latest version does not support go1.18 compilation!! Enhancements: - #8202: Indicate folders / devices where the remote end has not accepted sharing. - #8282: Change default log format to include date Other issues: - #8279: Simplify tests using `T.TempDir` 2022-04-21T13:57:00+00:00 caddy v2.5.0 caddy v2.5.0 2022-04-25T17:34:35+00:00 Caddy 2.5 introduces new features you'll love as well as a huge number of bug fixes and enhancements. Thank you to everyone who contributed! Feel free to ask on the [forum](https://caddy.community) if you have any questions or feedback. ## Highlights - **Reverse proxy:** [:sparkles: _Dynamic upstreams_](https://github.com/caddyserver/caddy/pull/4470), which is the ability to get the list of upstreams at every request (more specifically, every iteration in the proxy loop of every request) rather than just once at config-load time. Dynamic upstream modules can be plugged in to provide Caddy with the latest list of backends in real-time. Two standard modules have been implemented which can get upstreams from SRV and A/AAAA record lookups. - :warning: This deprecates the `lookup_srv` JSON field for upstreams (and `srv+` scheme prefix in the Caddyfile), which will be removed in the future. - **Automatic HTTPS:** Caddy will automatically try to get relevant certificates from the local [Tailscale](https://tailscale.com) instance (if running with permission to access the Tailscale socket). This makes services running on a Tailscale network [automatically available](https://github.com/caddyserver/caddy/pull/4541) over trusted HTTPS with Caddy. - **Tracing:** New [OpenTelemetry](https://opentelemetry.io/) integration with the [`tracing` handler module and associated `tracing` directive](https://github.com/caddyserver/caddy/pull/4361). - **Reverse proxy:** When using the response handlers, a new handler `copy_response` is available to copy the proxy's response back to the client, and `copy_response_headers` may be used to selectively copy header values from the proxy's response. - **API:** Added new endpoints `/pki/ca/<id>` and `/pki/ca/<id>/certificates` for getting information about Caddy's managed CAs, including the chain of root and intermediate certificates. ## Notable - **Reverse proxy:** The `X-Forwarded-Host` header will now be automatically set, along with `X-Forwarded-For` and `X-Forwarded-Proto`. - :warning: **Reverse proxy:** Incoming `X-Forwarded-*` headers will no longer be automatically trusted, to prevent spoofing. Now, `trusted_proxies` must be configured to specify a list of downstream proxies which are trusted to have sent good values. You only need to configure trusted proxies if Caddy is not the first server being connected to. For example, if you have Cloudflare in front of Caddy, then you should configure this with Cloudflare's [list of IP ranges](https://www.cloudflare.com/en-ca/ips/). - **Automatic HTTPS:** Revoked certificates will be automatically [replaced more reliably](https://github.com/caddyserver/certmagic/pull/166). - **Automatic HTTPS:** [Can now get certificates from _Managers_.](https://github.com/caddyserver/caddy/pull/4541) As opposed to _Issuers_ (such as the default ACME issuers) which give Caddy certificates to manage from a CSR, Managers give Caddy certificates to serve (rather than manage) during TLS handshakes. - **Automatic HTTPS:** A DNS challenge [domain override](https://github.com/caddyserver/caddy/issues/4071) can be configured to delegate the solving of the challenge to a different domain. - **Automatic HTTPS:** The DNS challenge propagation checks can now be delayed or disabled by setting `propagation_delay` or `propagation_timeout` to -1, respectively. - **Reverse proxy:** The default dial timeout for the HTTP transport [has been adjusted](https://github.com/caddyserver/caddy/pull/4436) down to `3s` (was `10s`), which should allow for more easily configuring load balancing retries. - **Logging:** HTTP access logs will now [render empty values](https://github.com/caddyserver/caddy/commit/7d5047c1f190421528695e1cc3a4ad71c97eb022) for often-sensitive HTTP headers such as Cookie, Authorization, and Proxy-Authorization. Logging such credentials is now [opt-in](https://github.com/caddyserver/caddy/commit/5bf0adad8748e96e10529d5fc5777afc9236a7b5) with the `log_credentials` global option in the Caddyfile, or the server's `logs > should_log_credentials` field in JSON. - **Logging:** Logs can now be filtered by [query string parameters](https://github.com/caddyserver/caddy/commit/bcac2beee7e419f8cdab2ed16f388d1af282a46b), [cookie values](https://github.com/caddyserver/caddy/commit/8887adb027982e844965b4707b8595cee5845d54), and [regular expressions](https://github.com/caddyserver/caddy/commit/789efa5deef53071b57479d37e4022bf372c4eef); and log values can be [hashed](https://github.com/caddyserver/caddy/commit/a1b417c832b4ab3dab9eaa9690e1d07672a949b8). These features are useful for redacting sensitive information. - **Logging:** Errors during request handling [will now be logged at `DEBUG` level](https://github.com/caddyserver/caddy/pull/4429) if the error was [handled via `errors` routes](https://github.com/caddyserver/caddy/pull/4584) (`handle_errors` in Caddyfile). - :warning: **Logging:** Removed the [deprecated](https://github.com/caddyserver/caddy/issues/4148) `common_log` field from HTTP access logs, and the `single_field` encoder. If you relied on this, you may use the [transform encoder plugin](https://github.com/caddyserver/transform-encoder) to encode logs in Common Log format. - :warning: **Logging:** The `remote_addr` field [has been replaced](https://github.com/caddyserver/caddy/commit/f55b123d63132e290789bcd07077375c76b6e1dd) by `remote_ip` and `remote_port` fields in HTTP access logs, which split up the two parts of the remote address. This improves ease of use for some tooling which only expect an IP address, without a port. - **HTTP server:** The [`vars` matcher](https://github.com/caddyserver/caddy/commit/ecac03cdcb6cceae743aac16faca7f32e5da1607) can now match on multiple possible values. - **HTTP server:** Requests [can now be assigned](https://github.com/caddyserver/caddy/commit/180ae0cc4843ecc3c7ddcb6e978ebfd474ed07f9) a random and unique UUID from the new `{http.request.uuid}` placeholder. - **HTTP server:** [New `http_redirect` listener wrapper](https://github.com/caddyserver/caddy/pull/4585) which can be used to redirect HTTP requests that come in on a server listening for HTTPS requests to be redirected to `https://`. - :warning: **Caddyfile:** [Deprecated paths in site addresses.](https://github.com/caddyserver/caddy/pull/4728) Prefer using path matchers within your site block instead. - **Caddyfile:** [New `default_bind` global option](https://github.com/caddyserver/caddy/pull/4531) lets you specify the default interface all sockets should bind to. - **Caddyfile:** [New `pki` global option](https://github.com/caddyserver/caddy/pull/4450) lets you configure the properties of the internal CAs managed by Caddy. - **Caddyfile:** [New `method` directive](https://github.com/caddyserver/caddy/pull/4528) allows rewriting the request method via Caddyfile. - :warning: **Caddyfile:** The `reverse_proxy` directive's `handle_response` subdirective has had its status replacement functionality [moved to a new `replace_status`](https://github.com/caddyserver/caddy/pull/4300) subdirective. This makes sure that the functionality of `handle_response` is not overloaded, and usage is clearer. - **Caddyfile**: The `map` directive [now casts outputs](https://github.com/caddyserver/caddy/commit/93c99f67342504efe9f6b58a734aaec3929fe785) to the appropriate scalar type if possible (int, float, bool). If you need to force a string, you may use double quotes or backticks https://github.com/caddyserver/caddy/pull/4643. - **Caddyfile**: [New `vars` directive](https://github.com/caddyserver/caddy/commit/79cbe7bfd06565d0e7ab0717119f78960ed54c08) allows setting some variables during request handling for later use in another handler or matcher. - **Caddyfile**: The Caddyfile adapter [is now stricter](https://github.com/caddyserver/caddy/pull/4655) about curly braces for block openers to try to prevent parsing ambiguities. - **Caddyfile**: The `caddy fmt` CLI command now has a [`--diff` option](https://github.com/caddyserver/caddy/pull/4695) which lets you visually see the formatting differences. - :warning: **Admin:** [Renamed](https://github.com/caddyserver/caddy/commit/bc447e307f195b80eeec0f6157e0d8e641af9155) experimental property `load_interval` :arrow_right: `load_delay` for clarification, and improved dynamic config loading. --- :shield: Thanks to [David Leadbeater](https://github.com/dgl) for reporting a security vulnerability related to HTTP methods and metrics cardinality, which was fixed in this release. ## New Contributors * @adamburgess made their first contribution in https://github.com/caddyserver/caddy/pull/4460 * @12f23eddde made their first contribution in https://github.com/caddyserver/caddy/pull/4444 * @rayjlinden made their first contribution in https://github.com/caddyserver/caddy/pull/4023 * @GallopingKylin made their first contribution in https://github.com/caddyserver/caddy/pull/4522 * @ForestJohnson made their first contribution in https://github.com/caddyserver/caddy/pull/4534 * @VojtechVitek made their first contribution in https://github.com/caddyserver/caddy/pull/4535 * @Ikke made their first contribution in https://github.com/caddyserver/caddy/pull/4544 * @YourTechBud made their first contribution in https://github.com/caddyserver/caddy/pull/4603 * @BitWuehler made their first contribution in https://github.com/caddyserver/caddy/pull/4597 * @ttys3 made their first contribution in https://github.com/caddyserver/caddy/pull/4572 * @crccw made their first contribution in https://github.com/caddyserver/caddy/pull/4596 * @andriikushch made their first contribution in https://github.com/caddyserver/caddy/pull/4361 * @renbou made their first contribution in https://github.com/caddyserver/caddy/pull/4654 * @cuishuang made their first contribution in https://github.com/caddyserver/caddy/pull/4702 ## Changelog * 2e46c2ac admin, reverseproxy: Stop timers if canceled to avoid goroutine leak (#4482) * 40b54434 admin: Enforce and refactor origin checking * b4bfa29b admin: Require identity for remote (fix #4478) * 32aad909 admin: Write proper status on invalid requests (#4569) (fix #4561) * ff137d17 caddyconfig: Support placeholders in HTTP loader * b47af6ef caddyfile: Copy input before parsing (fix #4422) * e90d7517 caddyfile: impove fmt warning message (#4444) * 5e5af50e caddyfile: make renew_interval option configurable (#4451) * ddbb234d caddyhttp: Always log handled errors at debug level (#4584) * 6b385a36 caddyhttp: Don't attempt to manage Tailscale certs * ecac03cd caddyhttp: Enhance vars matcher (#4433) * 6e6ce2be caddyhttp: Fix HTTP->HTTPS redir not preferring HTTPS port if ambiguous (#4530) * 3fe2c73d caddyhttp: Fix `MatchPath` sanitizing (#4499) * 44e5e9e4 caddyhttp: Fix test when /tmp/etc already exists (#4544) * 2bb8550a caddyhttp: Honor wildcard hosts in log SkipHosts (#4606) * 180ae0cc caddyhttp: Implement http.request.uuid placeholder (#4285) * 7d5047c1 caddyhttp: Log empty value for typical password headers * eead3373 caddyhttp: Log non-500 handler errors at debug level (#4429) * 5bf0adad caddyhttp: Make logging of credential headers opt-in (#4438) * 186fdba9 caddyhttp: Move HTTP redirect listener to an optional module (#4585) * 80d7a356 caddyhttp: Redirect HTTP requests on the HTTPS port to https:// (#4313) * bf380d00 caddyhttp: Reject absurd methods (#4538) * 850e1605 caddyhttp: Return HTTP 421 for mismatched Host header (#4023) * f55b123d caddyhttp: Split up logged remote address into IP and port (#4403) * ac14b64e caddyhttp: Support zone identifiers in remote_ip matcher (#4597) * a1c41210 caddypki: Minor tweak, don't use context pointer * 78e381b2 caddypki: Refactor /pki/ admin endpoints * c634bbe9 caddypki: Return error if no PEM data found * 9b7cdfa2 caddypki: Try to fix lint warnings * a79b4055 caddytls: Add internal Caddyfile `lifetime`, `sign_with_root` opts (#4513) * 77a77c02 caddytls: Add `propagation_delay`, support `propagation_timeout -1` (#4723) * 66de438a caddytls: Fix `MatchRemoteIP` provisoning with multiple CIDR ranges (#4522) * 57a708d1 caddytls: Support external certificate Managers (like Tailscale) (#4541) * d9b1d463 caddytls: dns_challenge_override_domain for challenge delegation (#4596) * 1a7a78a1 cmd: Print error if fmt overwrite fails (fix #4524) * bc447e30 core: Config LoadInterval -> LoadDelay for clarity * 7ea5b2a8 core: Config load interval only reloads if changed (#4603) * 7f364c77 core: Load config at interval instead of just once * a72acd21 core: Retry dynamic config load if config unchanged * ceef70db core: Retry dynamic config load if error or no-op (#4603) * acbee947 core: Revert 7f364c7; simplify dynamic config load * 64a3218f core: Simplify shared listeners, fix deadline bug * 8e5aafa5 fastcgi: Fix a TODO, prevent zap using reflection for logging env (#4437) * c8f2834b fastcgi: Protect against requests with null bytes in the path (#4614) * de490c7c fastcgi: Set SERVER_PORT to 80 or 443 depending on scheme (#4572) * 09ba9e99 fileserver: Add `pass_thru` Caddyfile option (#4613) * 15c95e9d fileserver: Canonical redir when whole path is stripped (#4549) * c8b5a816 fileserver: Fix handling of symlink sizes in directory listings (#4415) * e81369e2 fileserver: Move default browse template into a separate file (#4417) * 1e10f6f7 fileserver: browse: do not encode the paths in breadcrumbs and page title (#4410) * 78b5356f fileserver: do not double-escape paths (#4447) * 0de51593 go.mod: Revert version bump of CEL (#4587) * 6f9b6ad7 go.mod: Update smallstep/certificates, no longer need replace (#4475) * 4906b935 go.mod: Update smallstep/truststore, fix build on FreeBSD (#4473) * c1331534 go.mod: Update to latest smallstep/truststore, support FreeBSD (#4453) * ff74a0aa go.mod: Upgrade dependencies * e9dde230 headers: Fix `+` in Caddyfile to properly append rather than set (#4506) * 1b7ff5d7 httpcaddyfile: Add `default_bind` global option (#4531) * a8bb4a66 httpcaddyfile: Add `{vars.*}` placeholder shortcut, reverse `vars` sort order (#4726) * 5a071568 httpcaddyfile: Add pki app `root` and `intermediate` cert/key config (#4514) * 3a1e0dbf httpcaddyfile: Deprecate paths in site addresses; use zap logs (#4728) * 26d633ba httpcaddyfile: Disabling OCSP stapling for both managed and unmanaged (#4589) * 93a7a45e httpcaddyfile: Fix incorrect handling of IPv6 bind addresses (#4532) * 81ee34e9 httpcaddyfile: Fix sorting edgecase for nested `handle_path` (#4477) * 4b9849c7 httpcaddyfile: Support configuring `pki` app names via global options (#4450) * 5bd96a6a httpcaddyfile: Support explicitly turning off `strict_sni_host` (#4592) * c921e082 logging: Add `roll_local_time` Caddyfile option (#4583) * 0eb0b60f logging: Remove common_log field and single_field encoder (#4149) (#4282) * 249adc1c logging: Support turning off roll compression via Caddyfile (#4505) * 8887adb0 logging: add a filter for cookies (#4425) * bcac2bee logging: add a filter for query parameters (#4424) * 789efa5d logging: add a regexp filter (#4426) * a1b417c8 logging: add support for hashing data (#4434) * eb891d46 metrics: Enforce smaller set of method labels * c04d24ca pki: Avoid provisioning the `local` CA when not necessary (#4463) * bbad6931 pki: Implement API endpoints for certs and `caddy trust` (#4443) * 9ee68c1b reverseproxy: Adjust defaults, document defaults (#4436) * 7557d1d9 reverseproxy: Avoid returning a `nil` error during GetClientCertificate (#4550) * ab045592 reverseproxy: Dynamic upstreams (with SRV and A/AAAA support) (#4470) * 5333c352 reverseproxy: Fix incorrect `health_headers` Caddyfile parsing (#4485) * c50094fc reverseproxy: Implement trusted proxies for `X-Forwarded-*` headers (#4507) * f5e10494 reverseproxy: Make shallow-ish clone of the request (#4551) * 87a1f228 reverseproxy: Move status replacement intercept to `replace_status` (#4300) * d058dee1 reverseproxy: Refactor dial address parsing, augment command parsing (#4616) * c7d6c4cb reverseproxy: copy_response and copy_response_headers for handle_response routes (#4391) * bcb7a19c rewrite: Add `method` Caddyfile directive (#4528) * 1feb6595 rewrite: Fix a double-encode issue when using the `{uri}` placeholder (#4516) * 6cadb60f templates: Document .OriginalReq * 1d0425b2 templates: Elaborate on what's supported by the markdown function (#4564) * a6199cf8 templates: Fix docs for .Args * ec14ccdd templates: fix inconsistent nested includes (#4452) * d0b608af tracing: New OpenTelemetry module (#4361) * 134b8056 caddyfile: Prevent bad block opening tokens (#4655) * c5fffb4a caddyfile: Support for raw token values; improve `map`, `expression` (#4643) * b82e22b4 caddyhttp: retain all values of vars matcher when specified multiple times (#4629) * bc15b4b0 caddypki: Load intermediate for signing on-the-fly (#4669) * 6512832f cmd: Add `--diff` option for `caddy fmt` (#4695) * 30b6d1f4 cmd: Enhance .env (dotenv) file parsing * 22d8edb9 cmd: Fix defaulting admin address if empty in config, fixes `reload` (#4674) * c2327161 cmd: Set Origin header properly on API requests * d06d0e79 go.mod: Upgrade CertMagic to v0.16.0 * bf6a1b75 go.mod: Upgrade some dependencies * 79cbe7bf httpcaddyfile: Add 'vars' directive * a58f240d httpcaddyfile: Fix #4640 (auto-HTTPS edgecase) (#4661) * 7d229665 logging: Caddyfile support for `duration_format` (#4684) * 55b4c12e map: Evaluate placeholders in output vals (#4650) * 93c99f67 map: Support numeric and bool types with Caddyfile * 3d616e8c requestbody: Return HTTP 413 (fix #4558) * 3e3bb002 reverseproxy: Add `_ms` placeholders for proxy durations (#4666) * 2196c92c reverseproxy: Don't clear name in SRV upstreams * dc4d1473 reverseproxy: Expand SRV/A addrs for cache key * b8dbecb8 reverseproxy: Include port in A upstreams cache * e4ce40f8 reverseproxy: Sync up `handleUpgradeResponse` with stdlib (#4664) * afca2421 staticfiles: Expand placeholder for index files (#4679) * 00234c8a templates: Switch to `BurntSushi/toml` (#4700) **Full Changelog**: https://github.com/caddyserver/caddy/compare/v2.4.6...v2.5.0 2022-04-25T17:34:35+00:00 PeekabooAV v2.1 PeekabooAV v2.1 2022-04-26T08:20:37+00:00 - Peekaboo now provides a REST API. The old UNIX domain socket is gone and there's no longer a long-lived client connection providing a summary report on multiple samples. Samples are now submitted individually, yielding a job ID for subsequent attempts at retrieving a report. Both inputs and outputs of the API are JSON. The AMaViS plugin and peekaboo-util are updated to match. - Embedded Cuckoo mode and python2 support are removed. - Breaking change: Equality operators in expressions using regexes do now need to match the whole string up to the end. - New database schema version 9. Removes tables PeekabooMetadata and AnalysisJournal, and adds field analysis\_time as well as state to SampleInfo. - Generic rules can now make use of the new analyser `knownreport` - Introduce cortexreport toolbox analyser to connect to Cortex by TheHive. There already are a few sub analysers that can be used. - Reduce amount of data copied from Cuckoo reports for memory efficiency and security reasons. Reduces the amount of information available in Peekaboo processing failure dumps as well. URL to access original report via Cuckoo API is provided instead. - The CortexAnalyser or more precisely every CortexAnalyser can now access domain, hash, and ip artifacts from within the Generic rules. - FileInfoAnalyzerReport has new attibutes md5sum, sha256sum, and ssdeepsum (now don't get to excited, ssdeep hashes can only be used as strings) - Input validation of reports adds a new pip requirement: schema - Availability of external resources, particularly Cuckoo and Cortex APIs is no longer checked at startup. Lack of availability is reported as individual job failure. - PID file is no longer created by default (but can be re-enabled by specifying a path). 2022-04-26T08:20:37+00:00 syncthing v1.20.0-rc.3 syncthing v1.20.0-rc.3 2022-04-26T11:55:37+00:00 Bugfixes: - #7855: Watching for changes using too many system resources on BSDs (kqueue) - #8020: Dangling symlink prevents filesystem watcher on Solaris - #8232: Missing include file endlessly spams log - #8258: The latest version does not support go1.18 compilation!! - #8261: TypeError: Cannot read properties of null (reading 'status') in the GUI after killing Syncthing Enhancements: - #8202: Indicate folders / devices where the remote end has not accepted sharing. - #8282: Change default log format to include date Other issues: - #8279: Simplify tests using `T.TempDir` 2022-04-26T11:55:37+00:00 dalton v3.2.2 dalton v3.2.2 2022-04-28T14:34:15+00:00 - Added explicit requirement for `Jinja2==3.0.3` 2022-04-28T14:34:15+00:00 maltrail 0.45 maltrail 0.45 2022-04-30T22:11:05+00:00 Start-of-month release 2022-04-30T22:11:05+00:00 syncthing v1.20.0 syncthing v1.20.0 2022-05-03T15:24:19+00:00 Bugfixes: - #7855: Watching for changes using too many system resources on BSDs (kqueue) - #8020: Dangling symlink prevents filesystem watcher on Solaris - #8232: Missing include file endlessly spams log - #8258: The latest version does not support go1.18 compilation!! - #8261: TypeError: Cannot read properties of null (reading 'status') in the GUI after killing Syncthing Enhancements: - #8202: Indicate folders / devices where the remote end has not accepted sharing. - #8282: Change default log format to include date Other issues: - #8279: Simplify tests using `T.TempDir` 2022-05-03T15:24:19+00:00 syncthing v1.20.1 syncthing v1.20.1 2022-05-04T16:32:31+00:00 Bugfixes: - #8320: Spurious failure of new connections ("unknown (newer?) version of the protocol" etc) 2022-05-04T16:32:31+00:00 rocksdb v7.2.2 rocksdb v7.2.2 2022-05-05T22:33:28+00:00 ## 7.2.2 (2022-04-28) ### Bug Fixes * Fixed a bug in async_io path where incorrect length of data is read by FilePrefetchBuffer if data is consumed from two populated buffers and request for more data is sent. ## 7.2.1 (2022-04-26) ### Bug Fixes * Fixed a bug where RocksDB could corrupt DBs with `avoid_flush_during_recovery == true` by removing valid WALs, leading to `Status::Corruption` with message like "SST file is ahead of WALs" when attempting to reopen. * RocksDB calls FileSystem::Poll API during FilePrefetchBuffer destruction which impacts performance as it waits for read requets completion which is not needed anymore. Calling FileSystem::AbortIO to abort those requests instead fixes that performance issue. ## 7.2.0 (2022-04-15) ### Bug Fixes * Fixed bug which caused rocksdb failure in the situation when rocksdb was accessible using UNC path * Fixed a race condition when 2PC is disabled and WAL tracking in the MANIFEST is enabled. The race condition is between two background flush threads trying to install flush results, causing a WAL deletion not tracked in the MANIFEST. A future DB open may fail. * Fixed a heap use-after-free race with DropColumnFamily. * Fixed a bug that `rocksdb.read.block.compaction.micros` cannot track compaction stats (#9722). * Fixed `file_type`, `relative_filename` and `directory` fields returned by `GetLiveFilesMetaData()`, which were added in inheriting from `FileStorageInfo`. * Fixed a bug affecting `track_and_verify_wals_in_manifest`. Without the fix, application may see "open error: Corruption: Missing WAL with log number" while trying to open the db. The corruption is a false alarm but prevents DB open (#9766). * Fix segfault in FilePrefetchBuffer with async_io as it doesn't wait for pending jobs to complete on destruction. * Fix ERROR_HANDLER_AUTORESUME_RETRY_COUNT stat whose value was set wrong in portal.h * Fixed a bug for non-TransactionDB with avoid_flush_during_recovery = true and TransactionDB where in case of crash, min_log_number_to_keep may not change on recovery and persisting a new MANIFEST with advanced log_numbers for some column families, results in "column family inconsistency" error on second recovery. As a solution the corrupted WALs whose numbers are larger than the corrupted wal and smaller than the new WAL will be moved to archive folder. * Fixed a bug in RocksDB DB::Open() which may creates and writes to two new MANIFEST files even before recovery succeeds. Now writes to MANIFEST are persisted only after recovery is successful. ### New Features * For db_bench when --seed=0 or --seed is not set then it uses the current time as the seed value. Previously it used the value 1000. * For db_bench when --benchmark lists multiple tests and each test uses a seed for a RNG then the seeds across tests will no longer be repeated. * Added an option to dynamically charge an updating estimated memory usage of block-based table reader to block cache if block cache available. To enable this feature, set `BlockBasedTableOptions::reserve_table_reader_memory = true`. * Add new stat ASYNC_READ_BYTES that calculates number of bytes read during async read call and users can check if async code path is being called by RocksDB internal automatic prefetching for sequential reads. * Enable async prefetching if ReadOptions.readahead_size is set along with ReadOptions.async_io in FilePrefetchBuffer. * Add event listener support on remote compaction compactor side. * Added a dedicated integer DB property `rocksdb.live-blob-file-garbage-size` that exposes the total amount of garbage in the blob files in the current version. * RocksDB does internal auto prefetching if it notices sequential reads. It starts with readahead size `initial_auto_readahead_size` which now can be configured through BlockBasedTableOptions. * Add a merge operator that allows users to register specific aggregation function so that they can does aggregation using different aggregation types for different keys. See comments in include/rocksdb/utilities/agg_merge.h for actual usage. The feature is experimental and the format is subject to change and we won't provide a migration tool. * Meta-internal / Experimental: Improve CPU performance by replacing many uses of std::unordered_map with folly::F14FastMap when RocksDB is compiled together with Folly. * Experimental: Add CompressedSecondaryCache, a concrete implementation of rocksdb::SecondaryCache, that integrates with compression libraries (e.g. LZ4) to hold compressed blocks. ### Behavior changes * Disallow usage of commit-time-write-batch for write-prepared/write-unprepared transactions if TransactionOptions::use_only_the_last_commit_time_batch_for_recovery is false to prevent two (or more) uncommitted versions of the same key in the database. Otherwise, bottommost compaction may violate the internal key uniqueness invariant of SSTs if the sequence numbers of both internal keys are zeroed out (#9794). * Make DB::GetUpdatesSince() return NotSupported early for write-prepared/write-unprepared transactions, as the API contract indicates. ### Public API changes * Exposed APIs to examine results of block cache stats collections in a structured way. In particular, users of `GetMapProperty()` with property `kBlockCacheEntryStats` can now use the functions in `BlockCacheEntryStatsMapKeys` to find stats in the map. * Add `fail_if_not_bottommost_level` to IngestExternalFileOptions so that ingestion will fail if the file(s) cannot be ingested to the bottommost level. * Add output parameter `is_in_sec_cache` to `SecondaryCache::Lookup()`. It is to indicate whether the handle is possibly erased from the secondary cache after the Lookup. 2022-05-05T22:33:28+00:00 caddy v2.5.1 caddy v2.5.1 2022-05-06T17:23:16+00:00 This is a minor patch release that fixes some bugs and also enhances `reverse_proxy` with capabilities that weren't ready in time for v2.5.0. ### Highlights - Fixed regression in Unix socket admin endpoints. - Fixed regression in `caddy trust` commands. - Hash-based load balancing policies (ip_hash, uri_hash, header, and cookie) use an improved highest-random-weight (HRW) algorithm for increased consistency. The new rendezvous hash will ensure a client or request is _consistently_ mapped to a particular upstream even if the list of upstreams changes. - The reverse proxy is now able to rewrite the method and URI on its internal copy of the request that goes to the upstream. Combined with new `handle_response` capabilities, this enables the reverse proxy to fire off ["pre-check requests"](https://github.com/caddyserver/caddy/pull/4739) (for lack of a better term) to make routing decisions based on the results of that call. This enables a commonly-emerging pattern called _forward authentication_ wherein a backend is queried to assess a client's authorization to be proxied. The [full, verbose config for this](https://github.com/caddyserver/caddy/pull/4739#issuecomment-1113901951) is very flexible but tedious, so we made a new wrapper directive called `forward_auth` that eliminates the boilerplate (very similar to the [`php_fastcgi` directive](https://caddyserver.com/docs/caddyfile/directives/php_fastcgi#expanded-form)): ``` forward_auth authelia:9091 { uri /api/verify?rd=https://auth.example.com copy_headers Remote-User Remote-Groups Remote-Name Remote-Email } ``` This works with authentication providers like Authelia, and more. ## What's Changed * caddypki: Fix `caddy trust` command to use the correct API endpoint by @francislavoie in https://github.com/caddyserver/caddy/pull/4730 * reverseproxy: Improve hashing LB policies with HRW by @mholt in https://github.com/caddyserver/caddy/pull/4724 * Add missing backticks by @mahgoh in https://github.com/caddyserver/caddy/pull/4737 * caddyhttp: Improve listen addr error message for IPv6 by @francislavoie in https://github.com/caddyserver/caddy/pull/4740 * cmd: Fix unix socket addresses for admin API requests by @francislavoie in https://github.com/caddyserver/caddy/pull/4742 * logging: Use `RedirectStdLog` by @francislavoie in https://github.com/caddyserver/caddy/pull/4732 * logging: Implement rename filter, changes field key names by @francislavoie in https://github.com/caddyserver/caddy/pull/4745 * httpcaddyfile: Fix duplicate access log when debug is on by @francislavoie in https://github.com/caddyserver/caddy/pull/4746 * reverseproxy: Fix Caddyfile support for `replace_status` by @francislavoie in https://github.com/caddyserver/caddy/pull/4754 * templates: Add custom template function registration by @kroppt in https://github.com/caddyserver/caddy/pull/4757 * reverseproxy: Permit resolver addresses to not specify a port by @francislavoie in https://github.com/caddyserver/caddy/pull/4760 * caddyfile: Shortcut for `remote_ip` for private IP ranges by @francislavoie in https://github.com/caddyserver/caddy/pull/4753 * reverseproxy: Support performing pre-check requests by @francislavoie in https://github.com/caddyserver/caddy/pull/4739 * map: Prevent output destinations overlap with Caddyfile shorthands by @francislavoie in https://github.com/caddyserver/caddy/pull/4657 ## New Contributors * @mahgoh made their first contribution in https://github.com/caddyserver/caddy/pull/4737 ## Changelog * ec86a2f7 caddyfile: Shortcut for `remote_ip` for private IP ranges (#4753) * dcc98da4 caddyhttp: Improve listen addr error message for IPv6 (#4740) * d543ad1f caddypki: Fix `caddy trust` command to use the correct API endpoint (#4730) * 2e4c0915 cmd: Fix unix socket addresses for admin API requests (#4742) * af732151 httpcaddyfile: Fix duplicate access log when debug is on (#4746) * 0be3d995 logging: Implement rename filter, changes field key names (#4745) * 3017b245 logging: Use `RedirectStdLog` to capture more stdlib logs (#4732) * f7be0ee1 map: Prevent output destinations overlap with Caddyfile shorthands (#4657) * 4a223f52 reverseproxy: Fix Caddyfile support for `replace_status` (#4754) * 40b193fb reverseproxy: Improve hashing LB policies with HRW (#4724) * e7fbee8c reverseproxy: Permit resolver addresses to not specify a port (#4760) * f6900fcf reverseproxy: Support performing pre-check requests (#4739) * e84e19a0 templates: Add custom template function registration (#4757) * 3ab64838 templates: Add missing backticks in docs (#4737) **Full Changelog**: https://github.com/caddyserver/caddy/compare/v2.5.0...v2.5.1 2022-05-06T17:23:16+00:00 syncthing v1.20.2-rc.1 syncthing v1.20.2-rc.1 2022-05-10T06:35:01+00:00 Bugfixes: - #7289: TCP port 0 is announced in the LAN beacon - #8314: https://relays.syncthing.net/endpoint contains non-valid URLs with spaces Enhancements: - #8264: Include default ignore patterns in the GUI's advanced configuration editor 2022-05-10T06:35:01+00:00 TheHive 4.1.20 TheHive 4.1.20 2022-05-16T12:06:25+00:00 ## [4.1.20](https://github.com/TheHive-Project/TheHive/milestone/90) (2022-05-16) **Implemented enhancements:** - [Enhancement] Improve artifact creation from responder operation [\#2383](https://github.com/TheHive-Project/TheHive/issues/2383) - [Enhancement] Accept operations in analyzer reports [\#2384](https://github.com/TheHive-Project/TheHive/issues/2384) - [Enhancement] Add queries to filter observables based on the type of object it belongs to [\#2385](https://github.com/TheHive-Project/TheHive/issues/2385) **Fixed bugs:** - [Bug] Unable to use AWS S3 as storage backend [\#2316](https://github.com/TheHive-Project/TheHive/issues/2316) - [Bug] Typo on migration elasticsearch http config [\#2374](https://github.com/TheHive-Project/TheHive/issues/2374) 2022-05-16T12:06:25+00:00 lynis 3.0.8 lynis 3.0.8 2022-05-17T13:10:32+00:00 ### Added - MALW-3274 - Detect McAfee VirusScan Command Line Scanner - PKGS-7346 Check Alpine Package Keeper (apk) - PKGS-7395 Check Alpine upgradeable packages - EOL for Alpine Linux 3.14 and 3.15 ### Changed - AUTH-9408 - Check for pam_faillock as well (replacement for pam_tally2) - FILE-7524 - Test enhanced to support symlinks - HTTP-6643 - Support ModSecurity version 2 and 3 - KRNL-5788 - Only run relevant tests and improved logging - KRNL-5820 - Additional path for security/limits.conf - KRNL-5830 - Check for /var/run/needs_restarting (Slackware) - KRNL-5830 - Add a presence check for /boot/vmlinuz - PRNT-2308 - Bugfix that prevented test from storing values correctly - Extended location of PAM files for AARCH64 - Some messages in log improved 2022-05-17T13:10:32+00:00 OpenTAXII 0.4.0 OpenTAXII 0.4.0 2022-05-20T11:55:43+00:00 Changelog ========= 0.4.0 (2022-05-20) ------------------ * Move next_param handling into `OpenTAXII2PersistenceAPI` 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-05-20T11:55:43+00:00 syncthing v1.20.2-rc.2 syncthing v1.20.2-rc.2 2022-05-24T08:16:12+00:00 Bugfixes: - #7289: TCP port 0 is announced in the LAN beacon - #8314: https://relays.syncthing.net/endpoint contains non-valid URLs with spaces - #8355: Upgrading from v1.19.2 to v1.20.x now requires chmod for syncing files Enhancements: - #8264: Include default ignore patterns in the GUI's advanced configuration editor - #8310: Indicate folders / devices where the remote end is paused. 2022-05-24T08:16:12+00:00 OpenTAXII 0.5.0 OpenTAXII 0.5.0 2022-05-24T11:54:11+00:00 Changelog ========= 0.5.0 (2022-05-24) ------------------ * Add support for publicly readable taxii 2 collections 0.4.0 (2022-05-20) ------------------ * Move next_param handling into `OpenTAXII2PersistenceAPI` 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-05-24T11:54:11+00:00 Lookyloo v1.12.0 Lookyloo v1.12.0 2022-05-24T13:33:18+00:00 # New Features ## Playwright The captures are now made via [Playwright](https://playwright.dev/) instead of [Splash](https://github.com/scrapinghub/splash). It is a major improvement as Playwright uses actual up-to-date browsers, in headless mode (instead of qt-webkit from ~2016). You can read more about the research that lead to this change [in the discussion](https://github.com/Lookyloo/lookyloo/discussions/243). The main other advantages of using playwright are the following: * Easier to install: it doesn't requires Docker in order to use Splash * Much better control of what happen in the browser while capturing: Playwright makes it extremely simple to instrument everything in the browsers. The capturing module already tries to solve reCaptcha if it detects it on the page. The capture is made by a [standalone](https://github.com/Lookyloo/PlaywrightCapture) python module that you can use in your own tools if you wish to. ## De-duplication If the exact same capture is triggered multiple times within 5 min, it is skipped and the requestor is redirected to the capture done before. # Fixes * Avoid discarding a capture on network error: when a redirect is broken down the line, we keep the chain up to that point * Issue when the MISP was submitted as un-published * [Docker] Properly handle archiving * [Docker] Init SRI hashes # Changes * Improve subsequent capture template on long URLs * Improve view of the capture page on small-ish screens * General maintenance and code cleanup * Improvement in the tree generation on edge cases * Bump JS/CSS libraries * Update bundled-in User-Agent file * Use pydeep2, comes with a bundled-in libfuzzy, easier to install. 2022-05-24T13:33:18+00:00 osquery 5.3.0 osquery 5.3.0 2022-05-24T20:33:25+00:00 <a name="5.3.0"></a> ## [5.3.0](https://github.com/osquery/osquery/releases/tag/5.3.0) [Git Commits](https://github.com/osquery/osquery/compare/5.2.3...5.3.0) osquery 5.3.0 brings several table improvements and bugfixes. Worth mentioning also the deprecation of the `smart_drive_info` table and the new warning added when incorrectly configuring a CLI only flag via the config file. In the next release CLI only flags will not be configurable through the config file or refresh anymore. This release represents commits from 15 contributors! Thank you all. ### Deprecation Notices - Deprecate unmaintainable legacy table, `smart_drive_info` [#7464](https://github.com/osquery/osquery/issues/7464) ### New Features - Add the option `tls_disable_status_log` to prevent status logs from being sent via TLS [#7550](https://github.com/osquery/osquery/pull/7550) - Add SQLite function `in_cidr_block` to check if IPv4/v6 addresses are within the supplied CIDR block [#7563](https://github.com/osquery/osquery/pull/7563) ### Table Changes - Add the `admindir` column to the `deb_packages` table to parse package databases on different paths [#7549](https://github.com/osquery/osquery/pull/7549) - Implement and fix `wifi_networks` on macOS Big Sur and newer [#7503](https://github.com/osquery/osquery/pull/7503) - Add windows/darwin support to `npm_packages` [#7536](https://github.com/osquery/osquery/pull/7536) - Move `apt_sources` and `yum_sources` tables to linux only [#7537](https://github.com/osquery/osquery/pull/7537) - Add homebrew paths to the `python_packages` table [#7535](https://github.com/osquery/osquery/pull/7535) - Mark `wall_time` column in `osquery_schedule` as hidden [#7501](https://github.com/osquery/osquery/pull/7501) - Add new metrics and improve description of existing ones in `osquery_schedule` [#7438](https://github.com/osquery/osquery/pull/7438) - Add the `mirrorlist` column in the table `yum_sources` [#7479](https://github.com/osquery/osquery/pull/7479) - Implement `output_size` for `osquery_schedule` [#7436](https://github.com/osquery/osquery/pull/7436) - `deb_packages` table: Use additional instead of index for the `admindir` column [#7573](https://github.com/osquery/osquery/pull/7573) - `certificates` table: Add Linux support [#7570](https://github.com/osquery/osquery/pull/7570) - Add `translated` column to `processes` table to indicate whether the process is running under Apple Rosetta [#7507](https://github.com/osquery/osquery/pull/7507) - Add the "internet password" type to the macOS `keychain_items` table [#7576](https://github.com/osquery/osquery/pull/7576) - Add `original filename` column to `file` table on Windows [#7156](https://github.com/osquery/osquery/pull/7156) ### Bug Fixes - Fix watchdog not killing unhealthy worker/extension fast enough [#7474](https://github.com/osquery/osquery/pull/7474) - Fix the `test_http_server.py` `--persist` option [#7497](https://github.com/osquery/osquery/pull/7497) - Update`profile.py --leaks` for python3 [#7534](https://github.com/osquery/osquery/pull/7534) - Fixes osquery tls connections to aws kinesis when tls_server_certs is set [#7450](https://github.com/osquery/osquery/pull/7450) - Fix parsing issue when a backslash as the last character on sudoers file line [#7440](https://github.com/osquery/osquery/pull/7440) - Change the JSON of the results coming from an event scheduled query to an array [#7434](https://github.com/osquery/osquery/pull/7434) - Fix globToRegex truncating UTF16 characters [#7430](https://github.com/osquery/osquery/pull/7430) - Prevent hanging when the WMI server does not respond [#7429](https://github.com/osquery/osquery/pull/7429) - Fix `python_packages` table so that it lists python packages from any user Python installations [#7414](https://github.com/osquery/osquery/pull/7414) - Set string size limit on thrift protocol factory to prevent a crash [#7484](https://github.com/osquery/osquery/pull/7484) - Fix driver image path in `drivers` table [#7444](https://github.com/osquery/osquery/pull/7444) - Do not remove nonblocking flag when reading "special" files, to prevent hangs [#7530](https://github.com/osquery/osquery/pull/7530) - Fix crash due to interaction between distributed and config plugin [#7504](https://github.com/osquery/osquery/pull/7504) - bpf: Disable the BPF publisher in case of error [#7500](https://github.com/osquery/osquery/pull/7500) - Warn about setting CLI_FLAGs in the config [#7583](https://github.com/osquery/osquery/pull/7583) - Explicitly set context for the tables reading utmpx databases [#7578](https://github.com/osquery/osquery/pull/7578) - bpf: Improve socket event handling [#7446](https://github.com/osquery/osquery/pull/7446) - certificates: Refactor the OpenSSL utilities [#7581](https://github.com/osquery/osquery/pull/7581) - Fix shared_resources accessing uninitialized variables [#7600](https://github.com/osquery/osquery/pull/7600) ### Under the Hood improvements - Implement a performant cache for users and groups on Windows [#7516](https://github.com/osquery/osquery/pull/7516) - Replace WmiRequest constructor with static factory method to improve error handling and prevent crashes [#7489](https://github.com/osquery/osquery/pull/7489) - Remove redundant string conversion [#7603](https://github.com/osquery/osquery/pull/7603) ### Build - Fix DebPackages.test_sanity test when the `size` column is empty [#7569](https://github.com/osquery/osquery/pull/7569) - libs: Update libdpkg from version v1.19.0.5 to v1.21.7 [#7549](https://github.com/osquery/osquery/pull/7549) - CI: Restore some release checks [#7558](https://github.com/osquery/osquery/pull/7558) - Prevent ebpfpub linking against the system zlib [#7557](https://github.com/osquery/osquery/pull/7557) - Fix mdfind.test_sanity flaky behavior [#7533](https://github.com/osquery/osquery/pull/7533) - Enable fuzzing and Asan on Windows, enable Asan on macOS [#7470](https://github.com/osquery/osquery/pull/7470) - Update cppcheck to version 2.6.3 and skip analysis for third party code [#7455](https://github.com/osquery/osquery/pull/7455) - Change `cpu_info` test to expect *at least* one socket, not just one [#7490](https://github.com/osquery/osquery/pull/7490) - Fix third party libraries flags leaking to osquery targets [#7480](https://github.com/osquery/osquery/pull/7480) - Add third party libraries target [#7467](https://github.com/osquery/osquery/pull/7467) - Do not run clang-tidy on third party libraries [#7432](https://github.com/osquery/osquery/pull/7432) - CI: Create github workflow target to gate mergeability [#7427](https://github.com/osquery/osquery/pull/7427) - Fix some warnings about unrecognized special characters in the Windows event log test [#7478](https://github.com/osquery/osquery/pull/7478) - Change where the macOS Info.plist is generated [#7566](https://github.com/osquery/osquery/pull/7566) - Add OSQUERY_ENABLE_THREAD_SANITIZER to optionally enable TSan [#6997](https://github.com/osquery/osquery/pull/6997) - Add an option to specify a path to the openssl archive [#7559](https://github.com/osquery/osquery/pull/7559) - packs: Update reverse shell query pack to check for a valid remote_port [#7567](https://github.com/osquery/osquery/pull/7567) - Remove the test_daemon_sighup test [#7584](https://github.com/osquery/osquery/pull/7584) ### Documentation - docs: remove FreeBSD [#7508](https://github.com/osquery/osquery/pull/7508) - Pin Jinja2 ReadTheDocs dependency to 3.0.3 [#7533](https://github.com/osquery/osquery/pull/7533) - CHANGELOG 5.2.3 [#7571](https://github.com/osquery/osquery/pull/7571) - CHANGELOG 5.2.2 [#7447](https://github.com/osquery/osquery/pull/7447) - Bump mkdocs from 1.1.2 to 1.2.3 in /docs [#7457](https://github.com/osquery/osquery/pull/7457) - Replace OS X with macOS in table specs [#7587](https://github.com/osquery/osquery/pull/7587) - Update `osquery.example.conf` to omit the CLI only flags [#7595](https://github.com/osquery/osquery/pull/7595) 2022-05-24T20:33:25+00:00 OpenTAXII 0.6.0 OpenTAXII 0.6.0 2022-05-25T11:15:18+00:00 Changelog ========= 0.6.0 (2022-05-25 ------------------ * Add `public_discovery` option to taxii2 config * Add support for publicly readable taxii 2 api roots 0.5.0 (2022-05-24) ------------------ * Add support for publicly readable taxii 2 collections 0.4.0 (2022-05-20) ------------------ * Move next_param handling into `OpenTAXII2PersistenceAPI` 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-05-25T11:15:18+00:00 dalton v3.2.3 dalton v3.2.3 2022-05-26T14:51:26+00:00 - Added share link on job results page to recreate job automatically - Added PCAP drag and drop to UI when creating a new job 2022-05-26T14:51:26+00:00 OpenTAXII 0.7.0 OpenTAXII 0.7.0 2022-05-27T18:01:28+00:00 Changelog ========= 0.7.0 (2022-05-27) ------------------ * Nest taxii2 endpoints under `/taxii2/` 0.6.0 (2022-05-25) ------------------ * Add `public_discovery` option to taxii2 config * Add support for publicly readable taxii 2 api roots 0.5.0 (2022-05-24) ------------------ * Add support for publicly readable taxii 2 collections 0.4.0 (2022-05-20) ------------------ * Move next_param handling into `OpenTAXII2PersistenceAPI` 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-05-27T18:01:28+00:00 MISP v2.4.159 MISP v2.4.159 2022-05-30T17:05:38+00:00 We are pleased to announce the immediate availability of MISP v2.4.159. This releases includes many improvements, bugs fixing and improvement concerning performance on large dataset. ![](https://www.misp-project.org/img/blog/graph-syria.png) # Performance Improvements - [DB] Add MysqlExtended DboSource to support index query hints. - [Query] Add new setting to disable taxonomy checks when browsing data. - We discovered that some MISP users are still using slow file-based session handling in PHP. Now, we added in the diagnostic, if session is file based. We recommend everyone to use the Redis session. - Many additional speed-up and faster functions in the MISP internals. - Reduce memory usage when generating all correlations. # Improvements - [Feed] Allow option to disable correlations for all events coming from a feed. This can be useful when correlation requires to be disabled for an imported feed. - [UI] Allow to upload MISP event by pasting data to textarea in addition to the file upload. - An optional feature `clusters:attachMultipleClusters` is now available to allow the mirroring of attribute clusters to event. - [auditlog] Support for fetch event changes from specific time. - [UI] Allow to filter attributes from Related Events box. - [UI] Allow to filter attributes from warninglist box. - [UI] Many UI improvements to make the interface easier to read. - [UI] Disable correlation checkbox for non correlating types. - [STIX 2 import] Better Galaxies parsing by looking for the ATT&CK technique id. - [API] Enable sharing group filter for Event controller not just attribute. # Fixes - [STIX] Avoiding non RFC-4122 UUIDs to be imported (and therefore skipped) - [STIX 1 import] Save process network connections. - [STIX 1 import] Fixed galaxy tag_names fetching from TTP names. # Knowledge Bases ## MISP Taxonomies - [dga] First version of the DGA taxonomy based on https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_plohmann.pdf. - GrayZone of Active Defense, originaly published by Washington University, v2 created and updated by DCG420 - Various fixes to existing taxonomies. ## MISP Objects Template - A new PaloAlto Threat Event object template has been added. - A updated security playbook has been added. - A new ransom negotiation object has been added. - An improved Passive SSH template object. - Various fixes and improvements to different object templates such as email, virustotal-submissions and others. ## MISP Galaxy - Improved Cryptominers galaxy. - Improved backdoors galaxy. - Threat Actor galaxy updated and extended with new threat-actors. - MISP Galaxy updated for MITRE ATT&CK v11.2. # Acknowledgement We would like to thank all the [contributors](https://www.misp-project.org/contributors), reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in [misp-objects](https://www.misp-project.org/objects.html), [misp-taxonomies](https://www.misp-project.org/taxonomies.html) and [misp-galaxy](https://www.misp-project.org/galaxy.html). As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. Additional changelogs are available for [misp galaxy](https://www.misp-project.org/Changelog-misp-galaxy.txt), [misp-taxonomies](https://www.misp-project.org/Changelog-misp-taxonomies.txt), [misp-objects](https://www.misp-project.org/Changelog-misp-objects.txt) and [misp-modules](https://www.misp-project.org/Changelog-misp-modules.txt) 2022-05-30T17:05:38+00:00 HyperDbg v0.1.0 HyperDbg v0.1.0 2022-05-31T14:08:32+00:00 # HyperDbg v0.1 is released! **If you’re enjoying HyperDbg, don’t forget to give a star 🌟 on GitHub!** Please visit [Build & Install](https://docs.hyperdbg.org/getting-started/build-and-install) to configure the environment for running **HyperDbg**. Check out the [Quick Start](https://docs.hyperdbg.org/getting-started/quick-start) and [Frequently Asked Questions (FAQs)](https://docs.hyperdbg.org/getting-started/faq) to learn more. You can use the examples of [using the debugger](https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples) and the [script engine](https://docs.hyperdbg.org/commands/scripting-language/examples) to get started with **HyperDbg**. ## New Fearues * Advanced Hypervisor-based Kernel Mode Debugger [<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/getting-started/attach-to-hyperdbg/debug" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/getting-started/attach-to-hyperdbg/local-debugging" target="_blank">link</a>] * Classic EPT Hook (Hidden Breakpoint) [<a href="https://docs.hyperdbg.org/commands/extension-commands/epthook" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/features/vmm-module/design-of-epthook" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/hooking-any-function" target="_blank">link</a>] * Inline EPT Hook (Inline Hook) [<a href="https://docs.hyperdbg.org/commands/extension-commands/epthook2" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/features/vmm-module/design-of-epthook2" target="_blank">link</a>] * Monitor Memory For R/W (Emulating Hardware Debug Registers Without Limitation) [<a href="https://docs.hyperdbg.org/commands/extension-commands/monitor" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/features/vmm-module/design-of-monitor" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/monitoring-accesses-to-structures" target="_blank">link</a>] * SYSCALL Hook (Disable EFER & Handle #UD) [<a href="https://docs.hyperdbg.org/commands/extension-commands/syscall" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/features/vmm-module/design-of-syscall-and-sysret" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/intercepting-all-syscalls" target="_blank">link</a>] * SYSRET Hook (Disable EFER & Handle #UD) [<a href="https://docs.hyperdbg.org/commands/extension-commands/sysret" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/features/vmm-module/design-of-syscall-and-sysret" target="_blank">link</a>] * CPUID Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/cpuid" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/triggering-special-instructions" target="_blank">link</a>] * RDMSR Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/msrread" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/identifying-system-behavior" target="_blank">link</a>] * WRMSR Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/msrwrite" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/identifying-system-behavior" target="_blank">link</a>] * RDTSC/RDTSCP Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/tsc" target="_blank">link</a>] * RDPMC Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/pmc" target="_blank">link</a>] * VMCALL Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/vmcall" target="_blank">link</a>] * Debug Registers Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/dr" target="_blank">link</a>] * I/O Port (In Instruction) Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/ioin" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/triggering-special-instructions" target="_blank">link</a>] * I/O Port (Out Instruction) Hook & Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/ioout" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/triggering-special-instructions" target="_blank">link</a>] * MMIO Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/monitor" target="_blank">link</a>] * Exception (IDT < 32) Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/exception" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/features/vmm-module/design-of-exception-and-interrupt" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/identifying-system-behavior" target="_blank">link</a>] * External-Interrupt (IDT > 32) Monitor [<a href="https://docs.hyperdbg.org/commands/extension-commands/interrupt" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/features/vmm-module/design-of-exception-and-interrupt" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/events/identifying-system-behavior" target="_blank">link</a>] * Running Automated Scripts [<a href="https://docs.hyperdbg.org/commands/scripting-language/hyperdbg-scripts" target="_blank">link</a>] * Transparent-mode (Anti-debugging and Anti-hypervisor Resistance) [<a href="https://docs.hyperdbg.org/tips-and-tricks/considerations/transparent-mode" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/misc/defeating-anti-debug-and-anti-hypervisor-methods" target="_blank">link</a>] * Running Custom Assembly In Both VMX-root, VMX non-root (Kernel & User) [<a href="https://docs.hyperdbg.org/using-hyperdbg/prerequisites/how-to-create-an-action" target="_blank">link</a>] * Checking For Custom Conditions [<a href="https://docs.hyperdbg.org/using-hyperdbg/prerequisites/how-to-create-a-condition" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/debugger-internals/conditions" target="_blank">link</a>] * Process-specific & Thread-specific Debugging [<a href="https://docs.hyperdbg.org/commands/meta-commands/.process" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/commands/meta-commands/.thread" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/user-mode-debugging/examples/basics/switching-to-a-specific-process-or-thread" target="_blank">link</a>] * VMX-root Compatible Message Tracing [<a href="https://docs.hyperdbg.org/design/features/vmm-module/vmx-root-mode-compatible-message-tracing" target="_blank">link</a>] * Powerful Kernel Side Scripting Engine [<a href="https://docs.hyperdbg.org/commands/scripting-language" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/design/script-engine" target="_blank">link</a>] * Support To Symbols (Parsing PDB Files) [<a href="https://docs.hyperdbg.org/commands/meta-commands/.sympath" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/commands/meta-commands/.sym" target="_blank">link</a>] * Event Forwarding (#DFIR) [<a href="https://docs.hyperdbg.org/tips-and-tricks/misc/event-forwarding" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/commands/debugging-commands/output" target="_blank">link</a>] * Transparent Breakpoint Handler [<a href="https://docs.hyperdbg.org/commands/debugging-commands/bp" target="_blank">link</a>][<a href="https://docs.hyperdbg.org/using-hyperdbg/kernel-mode-debugging/examples/basics/setting-breakpoints-and-stepping-instructions" target="_blank">link</a>] * Various Custom Scripts [<a href="https://github.com/HyperDbg/scripts" target="_blank">link</a>] **Note**: community contributions are always welcomed and appreciated. If you plan to contribute a new feature, it's best to [discuss](https://github.com/HyperDbg/HyperDbg/discussions) it first. Bug fixes, tests, and documentation improvements are greatly appreciated. 2022-05-31T14:08:32+00:00 maltrail 0.46 maltrail 0.46 2022-05-31T22:11:06+00:00 Start-of-month release 2022-05-31T22:11:06+00:00 syncthing v1.20.2-rc.3 syncthing v1.20.2-rc.3 2022-06-01T05:34:59+00:00 Bugfixes: - #7289: TCP port 0 is announced in the LAN beacon - #8314: https://relays.syncthing.net/endpoint contains non-valid URLs with spaces - #8355: Upgrading from v1.19.2 to v1.20.x now requires chmod for syncing files Enhancements: - #8264: Include default ignore patterns in the GUI's advanced configuration editor - #8310: Indicate folders / devices where the remote end is paused. 2022-06-01T05:34:59+00:00 PyPCAPKit v0.16.0 PyPCAPKit v0.16.0 2022-06-01T05:35:59+00:00 ## What's Changed * revised entire project * type annotations * protocol redesign * linter compliance * added and revised tests (see #82, bugfix help wanted) * general bugfix in the project (#101 and many more) * reviewed and rearranged APIs, especially * added `pcapkit.corekit.multidict` module based on `Werkzeug` project * removed `validators` & some decorators from `pcapkit.utilities` * moved PCAP & aux protocols under `pcapkit.misc` * redesigned `Protocol` class to integrate parsing & construction at the same time, as well as better subclass protocol implementation experience * initialised PyPCAPKit Enhancement Proposals discussion channel (see #106) **Full Changelog**: https://github.com/JarryShaw/PyPCAPKit/compare/v0.15.5...v0.16.0 2022-06-01T05:35:59+00:00 OpenTAXII 0.8.0 OpenTAXII 0.8.0 2022-06-05T14:57:50+00:00 Changelog ========= 0.8.0 (2022-06-05) ------------------ * Nest details inside taxii2 job and allow counts without details 0.7.0 (2022-05-27) ------------------ * Nest taxii2 endpoints under `/taxii2/` 0.6.0 (2022-05-25) ------------------ * Add `public_discovery` option to taxii2 config * Add support for publicly readable taxii 2 api roots 0.5.0 (2022-05-24) ------------------ * Add support for publicly readable taxii 2 collections 0.4.0 (2022-05-20) ------------------ * Move next_param handling into `OpenTAXII2PersistenceAPI` 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-06-05T14:57:50+00:00 syncthing v1.20.2 syncthing v1.20.2 2022-06-07T10:13:46+00:00 Bugfixes: - #7289: TCP port 0 is announced in the LAN beacon - #8314: https://relays.syncthing.net/endpoint contains non-valid URLs with spaces - #8355: Upgrading from v1.19.2 to v1.20.x now requires chmod for syncing files Enhancements: - #8264: Include default ignore patterns in the GUI's advanced configuration editor - #8310: Indicate folders / devices where the remote end is paused. 2022-06-07T10:13:46+00:00 PyPCAPKit v0.16.1 PyPCAPKit v0.16.1 2022-06-08T19:59:55+00:00 ## What's Changed * changed `warnings.warn` calls to `pcapkit.utilities.warnings.warn` * revised missing optional dependency warnings (add for CLI & vendor) * updated version strings in repository * `make isort` **Full Changelog**: https://github.com/JarryShaw/PyPCAPKit/compare/v0.16.0...v0.16.1 2022-06-08T19:59:55+00:00 rocksdb v7.3.1 rocksdb v7.3.1 2022-06-10T23:08:05+00:00 ## 7.3.1 (06/08/2022) ### Bug Fixes * Fix a bug in WAL tracking. Before this PR (#10087), calling `SyncWAL()` on the only WAL file of the db will not log the event in MANIFEST, thus allowing a subsequent `DB::Open` even if the WAL file is missing or corrupted. * Fixed a bug for non-TransactionDB with avoid_flush_during_recovery = true and TransactionDB where in case of crash, min_log_number_to_keep may not change on recovery and persisting a new MANIFEST with advanced log_numbers for some column families, results in "column family inconsistency" error on second recovery. As a solution, RocksDB will persist the new MANIFEST after successfully syncing the new WAL. If a future recovery starts from the new MANIFEST, then it means the new WAL is successfully synced. Due to the sentinel empty write batch at the beginning, kPointInTimeRecovery of WAL is guaranteed to go after this point. If future recovery starts from the old MANIFEST, it means the writing the new MANIFEST failed. We won't have the "SST ahead of WAL" error. * Fixed a bug where RocksDB DB::Open() may creates and writes to two new MANIFEST files even before recovery succeeds. Now writes to MANIFEST are persisted only after recovery is successful. ## 7.3.0 (05/20/2022) ### Bug Fixes * Fixed a bug where manual flush would block forever even though flush options had wait=false. * Fixed a bug where RocksDB could corrupt DBs with `avoid_flush_during_recovery == true` by removing valid WALs, leading to `Status::Corruption` with message like "SST file is ahead of WALs" when attempting to reopen. * Fixed a bug in async_io path where incorrect length of data is read by FilePrefetchBuffer if data is consumed from two populated buffers and request for more data is sent. * Fixed a CompactionFilter bug. Compaction filter used to use `Delete` to remove keys, even if the keys should be removed with `SingleDelete`. Mixing `Delete` and `SingleDelete` may cause undefined behavior. * Fixed a bug in `WritableFileWriter::WriteDirect` and `WritableFileWriter::WriteDirectWithChecksum`. The rate_limiter_priority specified in ReadOptions was not passed to the RateLimiter when requesting a token. * Fixed a bug which might cause process crash when I/O error happens when reading an index block in MultiGet(). ### New Features * DB::GetLiveFilesStorageInfo is ready for production use. * Add new stats PREFETCHED_BYTES_DISCARDED which records number of prefetched bytes discarded by RocksDB FilePrefetchBuffer on destruction and POLL_WAIT_MICROS records wait time for FS::Poll API completion. * RemoteCompaction supports table_properties_collector_factories override on compaction worker. * Start tracking SST unique id in MANIFEST, which will be used to verify with SST properties during DB open to make sure the SST file is not overwritten or misplaced. A db option `verify_sst_unique_id_in_manifest` is introduced to enable/disable the verification, if enabled all SST files will be opened during DB-open to verify the unique id (default is false), so it's recommended to use it with `max_open_files = -1` to pre-open the files. * Added the ability to concurrently read data blocks from multiple files in a level in batched MultiGet. This can be enabled by setting the async_io option in ReadOptions. Using this feature requires a FileSystem that supports ReadAsync (PosixFileSystem is not supported yet for this), and for RocksDB to be compiled with folly and c++20. * Add FileSystem::ReadAsync API in io_tracing. ### Public API changes * Add rollback_deletion_type_callback to TransactionDBOptions so that write-prepared transactions know whether to issue a Delete or SingleDelete to cancel a previous key written during prior prepare phase. The PR aims to prevent mixing SingleDeletes and Deletes for the same key that can lead to undefined behaviors for write-prepared transactions. * EXPERIMENTAL: Add new API AbortIO in file_system to abort the read requests submitted asynchronously. * CompactionFilter::Decision has a new value: kRemoveWithSingleDelete. If CompactionFilter returns this decision, then CompactionIterator will use `SingleDelete` to mark a key as removed. * Renamed CompactionFilter::Decision::kRemoveWithSingleDelete to kPurge since the latter sounds more general and hides the implementation details of how compaction iterator handles keys. * Added ability to specify functions for Prepare and Validate to OptionsTypeInfo. Added methods to OptionTypeInfo to set the functions via an API. These methods are intended for RocksDB plugin developers for configuration management. * Added a new immutable db options, enforce_single_del_contracts. If set to false (default is true), compaction will NOT fail due to a single delete followed by a delete for the same key. The purpose of this temporay option is to help existing use cases migrate. * Introduce `BlockBasedTableOptions::cache_usage_options` and use that to replace `BlockBasedTableOptions::reserve_table_builder_memory` and `BlockBasedTableOptions::reserve_table_reader_memory`. * Changed `GetUniqueIdFromTableProperties` to return a 128-bit unique identifier, which will be the standard size now. The old functionality (192-bit) is available from `GetExtendedUniqueIdFromTableProperties`. Both functions are no longer "experimental" and are ready for production use. * In IOOptions, mark `prio` as deprecated for future removal. * In `file_system.h`, mark `IOPriority` as deprecated for future removal. * Add an option, `CompressionOptions::use_zstd_dict_trainer`, to indicate whether zstd dictionary trainer should be used for generating zstd compression dictionaries. The default value of this option is true for backward compatibility. When this option is set to false, zstd API `ZDICT_finalizeDictionary` is used to generate compression dictionaries. * Seek API which positions itself every LevelIterator on the correct data block in the correct SST file which can be parallelized if ReadOptions.async_io option is enabled. * Add new stat number_async_seek in PerfContext that indicates number of async calls made by seek to prefetch data. ### Bug Fixes * RocksDB calls FileSystem::Poll API during FilePrefetchBuffer destruction which impacts performance as it waits for read requets completion which is not needed anymore. Calling FileSystem::AbortIO to abort those requests instead fixes that performance issue. * Fixed unnecessary block cache contention when queries within a MultiGet batch and across parallel batches access the same data block, which previously could cause severely degraded performance in this unusual case. (In more typical MultiGet cases, this fix is expected to yield a small or negligible performance improvement.) ### Behavior changes * Enforce the existing contract of SingleDelete so that SingleDelete cannot be mixed with Delete because it leads to undefined behavior. Fix a number of unit tests that violate the contract but happen to pass. * ldb `--try_load_options` default to true if `--db` is specified and not creating a new DB, the user can still explicitly disable that by `--try_load_options=false` (or explicitly enable that by `--try_load_options`). * During Flush write or Compaction write/read, the WriteController is used to determine whether DB writes are stalled or slowed down. The priority (Env::IOPriority) can then be determined accordingly and be passed in IOOptions to the file system. 2022-06-10T23:08:05+00:00 cwe_checker v0.6 cwe_checker v0.6 2022-06-13T05:48:56+00:00 Version 0.6 contains improved abstract domains able to represent data more precise and more complete. Furthermore, the Pointer Inference analysis was reworked to be a bottom-up analysis and an additional function signature analysis step was added to the analysis pipeline. These improvements allow all analyses depending on the Pointer Inference to be both more precise and more complete. Other highlights include: - A new command line flag for analyzing bare-metal binaries. - The check for *CWE-78: Command line injections* was completely rewritten using abstract domains for strings. - The check for *CWE-119: Buffer Overflow* was completely rewritten and now emits additional data flow information in the JSON output to help with root cause analysis. - The check for *CWE-416: Use After Free* was completely rewritten and now emits additional data flow information in the JSON output to help with root cause analysis. See the CHANGES.md for more details. 2022-06-13T05:48:56+00:00 OpenTAXII 0.9.0 OpenTAXII 0.9.0 2022-06-13T14:44:11+00:00 Changelog ========= 0.9.0 (2022-06-13) ------------------ * Allow custom properties. This can be disabled by config option ``allow_custom_properties`` 0.8.0 (2022-06-05) ------------------ * Nest details inside taxii2 job and allow counts without details 0.7.0 (2022-05-27) ------------------ * Nest taxii2 endpoints under `/taxii2/` 0.6.0 (2022-05-25) ------------------ * Add `public_discovery` option to taxii2 config * Add support for publicly readable taxii 2 api roots 0.5.0 (2022-05-24) ------------------ * Add support for publicly readable taxii 2 collections 0.4.0 (2022-05-20) ------------------ * Move next_param handling into `OpenTAXII2PersistenceAPI` 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-06-13T14:44:11+00:00 syncthing v1.20.3-rc.1 syncthing v1.20.3-rc.1 2022-06-14T12:24:10+00:00 Bugfixes: - #8376: Chrome Autofill Breaks Authentication 2022-06-14T12:24:10+00:00 MONARC v2.11.1-p2 MONARC v2.11.1-p2 2022-06-15T06:56:04+00:00 2022-06-15T06:56:04+00:00 monarc-stats-service v0.5.0 monarc-stats-service v0.5.0 2022-06-20T12:27:47+00:00 - chg: [documentation] Updated documentation (9f38db11b382d8516fb71b60154aa0c7ba77004c); - chg: [API] CLIENT_REGISTRATION_OPEN is now set to True by default (b277436f81bbac1445822f0399dc348c5e283f70); - fix: [security] prevent the creation of new admin users (even by an admin) (257c16fed890bda8974594238a743f8afda0ff5a); - fix: replaced after_request by before_request for the API (88a276bf4d5f35e4e5da6ac065e1eb62f2892670); - small codebase fix for container image (82cdeaa714dbff14b9068c0b65c302ec4d02b3c4); - dockerfile and build pipeline (f8c663b61e1c8475d0f17060690d9920a1cc9e90); - updated Python dependencies. 2022-06-20T12:27:47+00:00 MONARC v2.12.0 MONARC v2.12.0 2022-06-20T12:44:13+00:00 ### New - [compliance scale](https://github.com/monarc-project/MonarcAppFO/discussions/439) - [metadata assets](https://github.com/monarc-project/MonarcAppFO/discussions/437) - [two-factor authentication](https://github.com/monarc-project/MonarcAppFO/discussions/442) - new build deployment is available and based on GitHub Actions ### Fix - [Stats provider] removed the leading slash in the URI ([e7dfba1](https://github.com/monarc-project/zm-client/commit/e7dfba1cf64322bc3e83630df6729b525d7d5c8d)) Details about upcoming related releases: https://github.com/orgs/monarc-project/projects/3 2022-06-20T12:44:13+00:00 monarc-stats-service v0.5.1 monarc-stats-service v0.5.1 2022-06-21T21:59:16+00:00 Changes ~~~~~~~ - [dependenvies] Updated Python dependencies. 2022-06-21T21:59:16+00:00 MONARC v2.12.1 MONARC v2.12.1 2022-06-22T07:39:21+00:00 2022-06-22T07:39:21+00:00 TheHive 4.1.21 TheHive 4.1.21 2022-06-22T11:21:10+00:00 ## [4.1.21](https://github.com/TheHive-Project/TheHive/milestone/91) (2022-06-22) **Fixed bugs:** - [Bug] S3 storage fails with old version of Minio [\#2388](https://github.com/TheHive-Project/TheHive/issues/2388) - [Bug ] Authentication Bypass Vulnerability [\#2391](https://github.com/TheHive-Project/TheHive/issues/2391) 2022-06-22T11:21:10+00:00 Lookyloo v1.13.0 Lookyloo v1.13.0 2022-06-26T16:06:50+00:00 # Maintenance and bug-fixes release All releases don't need to contain new features, sometimes, it is just some cleanup, and it is okay. * Properly handle exceptions in some edge cases (fixes in har2tree) * Properly display an error message if the capture fails * Use the same default User-Agent in when a capture is submitted via the API as via the web interface. * Cleanup some legacy code * Bump all dependencies (JS/CSS and Python) # Still, there is a new-ish thing We revamped the package generator, and it should be [more usable](https://github.com/Lookyloo/lookyloo/pkgs/container/lookyloo). If it is not, let us know! 2022-06-26T16:06:50+00:00 DC3-MWCP 3.7.0 DC3-MWCP 3.7.0 2022-06-28T13:39:42+00:00 2022-06-28T13:39:42+00:00 syncthing v1.20.3-rc.2 syncthing v1.20.3-rc.2 2022-06-29T06:28:24+00:00 Bugfixes: - #8369: Shared device names are missing from "Edit Folder -> Sharing" - #8376: Chrome Autofill Breaks Authentication - #8386: Ignore patterns with wildcard and non-ASCII characters don't work as expected Enhancements: - #8393: Warn if two devices are introducers to each other 2022-06-29T06:28:24+00:00 MONARC v2.12.2 MONARC v2.12.2 2022-06-29T07:20:25+00:00 2022-06-29T07:20:25+00:00 maltrail 0.47 maltrail 0.47 2022-06-30T22:11:05+00:00 Start-of-month release 2022-06-30T22:11:05+00:00 TheHive 4.1.22 TheHive 4.1.22 2022-07-01T15:00:56+00:00 ## [4.1.22](https://github.com/TheHive-Project/TheHive/milestone/93) (2022-07-01) **Implemented enhancements:** - [Enhancement] Add check on user role [\#2401](https://github.com/TheHive-Project/TheHive/issues/2401) **Fixed bugs:** - [Bug] Use dedicated stream topic for stream dispatcher subscription [\#2400](https://github.com/TheHive-Project/TheHive/issues/2400) 2022-07-01T15:00:56+00:00 monarc-stats-service v0.5.2 monarc-stats-service v0.5.2 2022-07-04T08:41:37+00:00 ## Changes - [API] patch on client now expects again a model from Namespace (client_ns). - Updated Python dependencies. - [documentation] Updated links to documentation. - [documentation] Updated information about installation. - [deployment] added docker-compose.yml file. ## Fix - [API] enable patch method for enabling/disabling stats sharing. 2022-07-04T08:41:37+00:00 MONARC v2.12.2-p1 MONARC v2.12.2-p1 2022-07-04T08:56:31+00:00 2022-07-04T08:56:31+00:00 osquery 5.4.0 osquery 5.4.0 2022-07-06T21:20:21+00:00 2022-07-06T21:20:21+00:00 GDPRDPIAT v3.0.1 GDPRDPIAT v3.0.1 2022-07-07T10:35:43+00:00 ## What's Changed * Bugfixes/surveyjs cdn by @simonarnell in https://github.com/simonarnell/GDPRDPIAT/pull/6 **Full Changelog**: https://github.com/simonarnell/GDPRDPIAT/compare/v3.0.0...v3.0.1 2022-07-07T10:35:43+00:00 MONARC v2.12.2-p2 MONARC v2.12.2-p2 2022-07-07T10:44:13+00:00 Fixed an issue with sortable 1.15.0 https://github.com/monarc-project/ng-client/commit/4d8e61e56edf7b858db68381f2b99389b3921866 2022-07-07T10:44:13+00:00 MONARC v2.12.2-p3 MONARC v2.12.2-p3 2022-07-07T12:42:37+00:00 2022-07-07T12:42:37+00:00 OpenTAXII 0.9.1 OpenTAXII 0.9.1 2022-07-11T20:28:28+00:00 Changelog ========= 0.9.1 (2022-07-11) ------------------ * Implement `raise_unauthorized` for taxii2, this was missing and lead to 500 errors. 0.9.0 (2022-06-13) ------------------ * Allow custom properties. This can be disabled by config option ``allow_custom_properties`` 0.8.0 (2022-06-05) ------------------ * Nest details inside taxii2 job and allow counts without details 0.7.0 (2022-05-27) ------------------ * Nest taxii2 endpoints under `/taxii2/` 0.6.0 (2022-05-25) ------------------ * Add `public_discovery` option to taxii2 config * Add support for publicly readable taxii 2 api roots 0.5.0 (2022-05-24) ------------------ * Add support for publicly readable taxii 2 collections 0.4.0 (2022-05-20) ------------------ * Move next_param handling into `OpenTAXII2PersistenceAPI` 0.3.0 (2022-04-13) ------------------ * Implement taxii2.1 support 0.3.0a4 (2022-04-13) -------------------- * Merge changes from 0.2.4 maintenance release 0.3.0a3 (2022-01-21) -------------------- * Fix bug that prevented booting with only taxii1 config (`#217 <https://github.com/eclecticiq/OpenTAXII/issues/217>`_ thanks `@azurekid <https://github.com/azurekid>`_ for the report) 0.3.0a2 (2021-12-27) -------------------- * Merge changes from 0.2.3 maintenance release 0.3.0a1 ------- * Add python 3.10 support 0.3.0a0 ------- * Enablement for future taxii2 implementation * Fix documentation build issues 0.2.4 (2022-04-13) ------------------ * Make sure werkzeug <2.1 and >=2.1 work correctly with auth system 0.2.3 (2021-12-22) ------------------ * Fix bug in multithreaded use of sqlite (`#210 <https://github.com/eclecticiq/OpenTAXII/issues/210>`_ thanks `@rohits144 <https://github.com/rohits144>`_ for the report) 0.2.2 (2021-11-05) ------------------ * Fix readthedocs build 0.2.1 (2021-11-03) ------------------ * Add tests for python 3.6, 3.7, 3.8, 3.9, pypy * Add tests for sqlite, mysql, mariadb, postgresql * Fix bug that broke ``delete_content_blocks`` when using mysql on sqlalchemy 1.3 * Docs: Add db schema diagram * Docs: Clarify how to get default data in a default (development) docker instance * Fix implicit routing in TAXII 1.1 Inboxes * Update jwt usage to pyjwt >= 2.0 (thanks `@SanyaKapoor <https://github.com/SanyaKapoor>`_) 0.2.0 (2020-06-30) ------------------ * Enforce UTC usage in datetime fields in SQL DB Persistence API. * `Fix for #114 <https://github.com/eclecticiq/OpenTAXII/issues/114>`_: reintroduce ``opentaxii-create-account`` CLI command. * `Fix for #153 <https://github.com/eclecticiq/OpenTAXII/issues/152>`_: check if user can modify a collection before advertising it over inbox service. * Multiple coding style fixes. * Various documentation updates. 0.1.12 (2019-03-06) ------------------- * Remove unnecessary print statements. 0.1.11 (2019-02-13) ------------------- * Make JSON logging consistent when the application is run via Gunicorn. * Set ``acceptable_destination`` key in status details instead of extended headers * Allow passing ``engine_parameters`` to ``SQLDatabaseAPI`` for those who want to customize SQLAlchemy engine parameters. * Require recent version of ``lxml`` for security reasons. * Various test and Docker infrastructure improvements. 0.1.10 (2018-06-03) ------------------- * Replace separate service/collection/account creation process with single ``opentaxii-sync-data`` CLI command. * Persistence and Auth APIs extended with missing CRUD methods, that are used by ``opentaxii-sync-data``. * Read/modify collection level ACL added. * DB models for default implementation of Persistence API and Auth API were changed. No automatic migration code is provided (sorry!), so upgrading might require manual DB migration. * Drop python2.7 from testing scope. * Various bug fixes and improvements. 0.1.9 (2017-06-19) ------------------ * `libtaxii <https://github.com/TAXIIProject/libtaxii>`_ dependency upgraded to 1.1.111. * Various bug fixes and improvements (thanks to `@bjigmp <https://github.com/bjigmp>`_, `@chorsley <https://github.com/chorsley>`_, `@rjprins <https://github.com/rjprins>`_). 0.1.8 (2017-02-21) ------------------ * Ability to enable/disable "huge trees" support in XML parser. Configuration property ``xml_parser_supports_huge_tree`` set to ``yes`` or ``true`` will disable security restrictions and force XML parser to support very deep trees and very long text content. * Adjust SQL Persistence API implemetation so it works smoothly with MySQL backend. * Use Python 3.5 instead of Python 3.4 for tests. 0.1.7 (2016-10-18) ------------------ * Minor fixes. * Dependencies were changed from hard-pinned to more flexible. * Example of production DB configuration added to docs. 0.1.6 (2016-06-01) ------------------ * Python 3.4 compatibility of the codebase. Tox configuration extended with python 3.4 environment run. * Flake8 full style compatibility. Flake8 check added to Tox configuration. * SQLAlchemy session scope issue fixed (related to `#38 <https://github.com/EclecticIQ/OpenTAXII/issues/38>`_). * `opentaxii-delete-blocks` CLI command added (related to `#45 <https://github.com/EclecticIQ/OpenTAXII/issues/45>`_). * `delete_content_blocks` method `added <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-6814849ac352b2b74132f8fa52e0ec4eR213>`_ to Persistence API. * Collection's name is `required <https://github.com/EclecticIQ/OpenTAXII/commit/dc6fddc27a98e8450c7e05e583b2bfb741f6e197#diff-ce3f7b939e5c540480ac655aef32c513R116>`_ to be unique in default SQL DB Auth API implementation. 0.1.5 (2016-03-15) ------------------ * Fix for the issue with persistence backend returning ``None`` instead of ``InboxMessage`` object 0.1.4 (2016-02-25) ------------------ * Hard-coded dependencies in ``setup.py`` removed. 0.1.3 (2016-02-25) ------------------ * Versions of dependencies are pinned. * Code adjusted for a new version of `anyconfig <https://pypi.python.org/pypi/anyconfig>`_ API. * Test for configuration loading added. 0.1.2 (2015-07-24) ------------------ * Docker configuration added. * Health check endpoint added. * Basic authentication support added. * Temporary workaround for `Issue #191 <https://github.com/TAXIIProject/libtaxii/issues/191>`_. * Method ``get_domain`` in Persistence API returns domain value configured for ``service_id``. If nothing returned, value set in configuration file will be used. * Performance optimisations. * Bug fixes and style improvements. 0.1.1 (2015-04-08) ------------------ * Alias for Root Logger added to logging configuration. * Context object in a request scope that holds account and token added. * Support for OPTIONS HTTP request to enable auto version negotiation added. * Documentation improved. 0.1.0 (2015-03-31) ------------------ * Initial release 2022-07-11T20:28:28+00:00 caddy v2.5.2 caddy v2.5.2 2022-07-12T19:06:28+00:00 This patch release fixes bugs, adds some new features, and makes worthwhile enhancements. We recommend everyone test and upgrade! Many improvements have been made to the `reverse_proxy` module. Highlights: - **New [`/adapt` admin endpoint](https://caddyserver.com/docs/api#post-adapt):** Use your installed config adapters via API in addition to the existing `caddy adapt` CLI command. - **New `Etag`/`If-Match` support for config API:** Safely update your config concurrently and avoid collisions by using [our unique Etag implementation](https://caddyserver.com/docs/api#concurrent-config-changes). - **Rename copied headers from reverse_proxy:** If you're using `handle_response`, you can more easily map headers to a different name for clients. - **Many HTTP matchers have been added to CEL:** You can now use the logic of our HTTP request matchers in CEL expressions. - **Notable bug fixes:** EAB reuse, various QUIC & HTTP/3 fixes, more specific HTTP status codes, various reverse proxy fixes. ## Changelog * 660c59b6 admin: Implement /adapt endpoint (close #4465) (#4846) * ad3a83fb admin: expect quoted ETags (#4879) * f259ed52 admin: support ETag on config endpoints (#4579) * 1498132e caddyhttp: Log error from CEL evaluation (fix #4832) * 0a14f97e caddytls: Make peer certificate verification pluggable (#4389) * 412dcc07 caddytls: Reuse issuer between PreCheck and Issue (#4866) * 499ad6d1 core: Micro-optim in run() (#4810) * c0f76e9e fileserver: Use safe redirects in file browser * 58e05cab forwardauth: Fix case when `copy_headers` is omitted (#4856) * 0b6f7643 forwardauth: Support renaming copied headers, block support (#4783) * 8bac134f go.mod: Bump up quic-go to v0.28.0, fixes for BC breaks (#4867) * 3d18bc56 go.mod: Update go-yaml to v3 * 56013934 go.mod: Update some dependencies * 8e6bc360 go.mod: Upgrade some dependencies * 53c4d788 headers: Only replace known placeholders (#4880) * 0bcd02d5 headers: Support wildcards for delete ops (close #4830) (#4831) * 58970cae httpcaddyfile: Add `{err.*}` placeholder shortcut (#4798) * b687d7b9 httpcaddyfile: Support multiple values for `default_bind` (#4774) * a9267791 reverseproxy: Add --internal-certs CLI flag #3589 (#4817) * aaf6794b reverseproxy: Add renegotiation param in TLS client (#4784) * 54d1923c reverseproxy: Adjust new TLS Caddyfile directive names (#4872) * 7f9b1f43 reverseproxy: Correct the `tls_server_name` docs (#4827) * c82fe911 reverseproxy: Dynamic ServerName for TLS upstreams (#4836) * d6bc9e0b reverseproxy: Err 503 if all upstreams unavailable * 98468af8 reverseproxy: Fix double headers in response handlers (#4847) * 25f10511 reverseproxy: Fix panic when TLS is not configured (#4848) * 5e729c1e reverseproxy: HTTP 504 for upstream timeouts (#4824) * f9b42c37 reverseproxy: Make TLS renegotiation optional * b6e96fa3 reverseproxy: Skip TLS for certain configured ports (#4843) * 57d27c1b reverseproxy: Support http1.1>h2c (close #4777) (#4778) * 9864b138 reverseproxy: api: Remove misleading 'healthy' value * 693e9b52 rewrite: Handle fragment before query (fix #4775) * 6891f7f4 templates: Add `humanize` function (#4767) * 9e760e2e templates: Documentation consistency (#4796) ## New Contributors * @nekohasekai made their first contribution in https://github.com/caddyserver/caddy/pull/4782 * @davidbgk made their first contribution in https://github.com/caddyserver/caddy/pull/4796 * @git001 made their first contribution in https://github.com/caddyserver/caddy/pull/4767 * @varianone made their first contribution in https://github.com/caddyserver/caddy/pull/4817 * @Gr33nbl00d made their first contribution in https://github.com/caddyserver/caddy/pull/4389 * @yaslama made their first contribution in https://github.com/caddyserver/caddy/pull/4784 * @kresike made their first contribution in https://github.com/caddyserver/caddy/pull/4836 * @TristonianJones made their first contribution in https://github.com/caddyserver/caddy/pull/4715 * @jhwz made their first contribution in https://github.com/caddyserver/caddy/pull/4579 **Full Changelog**: https://github.com/caddyserver/caddy/compare/v2.5.1...v2.5.2 2022-07-12T19:06:28+00:00 AIL-framework v4.2 AIL-framework v4.2 2022-07-16T08:40:47+00:00 ## AIL Framework version 4.2 released including typo squatting tracker, improved AIL2AIL sync, zerobinz fetcher and many bugs fixes v4.2 (2022-06-24) AIL Framework version 4.2 has been released including: - A new tracker for tracking potential typo squatted domains. This feature relies on the new [ail-typo-squatting ](https://github.com/ail-project/ail-typo-squatting) library which can be also used outside of AIL framework. This contribution is from @DavidCruciani - Many improvement and bugs fixed for the AIL2AIL sync. A huge thanks to @aaronkaplan from EU Directorate-General for Informatics (DIGIT) for support and tests during the long debugging sessions. - A new module for zerobinz to create an immediate crawler request if a zerobinz link appears in an item. The module can be used for other services with ephemeral content. Thanks to @gallypette for the contribution and the improvement ideas. - A new hosts detection module has been introduced. - Multiple bugs were fixed. ### Detailed Changes * [Tracker} Tracker_Typo_Squatting. [David Cruciani] * [v4.2] add v4.2 update. [Terrtia] * [investigation] fix investigation by user + delete an obj from all investigation. [Terrtia] * [install vitualenv] remove travis env. [Terrtia] * [Retro Hunt] add logs. [Terrtia] * [Retro Hunt] add logs. [Terrtia] * [Retro Hunt] add logs. [Terrtia] * [AIL2AIL Sync] update exchange format. [Terrtia] * [AIL2AIL Sync] update exchange format. [Terrtia] * [add Hosts module] [Terrtia] * [sync module] debug. [Terrtia] * [sync client] debug. [Terrtia] * [websockets client] bind client ip. [Terrtia] * [websocket server] add host and port config. [Terrtia] * [telegram importer] add username correlation. [Terrtia] * [UI subtype objs] get obj by subtype + name. [Terrtia] * [misp export] add username. [Terrtia] ### Fix * [typosquatting] remove unused import. [Thirion Aurélien] * [tracker] clean import. [Thirion Aurélien] * [tracker term] fix typosquatting key. [Thirion Aurélien] * [Typo] tracker typo. [David Cruciani] * [tracker] UI for other than typosquat. [David Cruciani] * [typo] UI. [David Cruciani] * [Language] fix cld3 import. [Terrtia] * [launcher] kill AIL_2_AIL screen. [Terrtia] * [cld3] enable cld3. [Terrtia] * [cld3 python3.10] temp disable cld3. [Terrtia] * [launcher] remove Travis test. [Terrtia] * [Retro Hunt] item directory. [Terrtia] * [Retro Hunt] item directory. [Terrtia] * [Retro Hunt] fix item directory. [Terrtia] * [AIL exchange mime-type] [Terrtia] * [Hosts module] module + launcher. [Terrtia] * [abstract module] exception traceback #145. [Terrtia] * [ui tag selector] force custom tags. [Terrtia] * [installer] remove old tor install. [Terrtia] * [sync module] fix redis tag queue. [Terrtia] * [sync module] fix tags filter. [Terrtia] * [sync client] debug. [Terrtia] * [sync client] debug. [Terrtia] * [sync module] debug. [Terrtia] * [websockets client] fix client bind. [Terrtia] * [websockets] remove size limit. [Terrtia] * [UI subtype objs] fix form. [Terrtia] * [misp config] https. [Thirion Aurélien] ### Other * Merge pull request #147 from ail-project/typo. [Thirion Aurélien] Integration of the typo-squatting tracker * Fix; [set tracker] missing function. [Thirion Aurélien] * Merge branch 'master' into typo. [David Cruciani] * Add: [tracker] typo-squatting. [David Cruciani] * Merge branch 'master' of github.com:ail-project/ail-framework. [Terrtia] * Merge pull request #146 from gallypette/master. [Thirion Aurélien] add: [modules] zerobinz * Add: [modules] zerobinz. [huynenjl@gmail.com] * Merge branch 'master' of github.com:ail-project/ail-framework. [Terrtia] 2022-07-16T08:40:47+00:00 rocksdb v7.4.3 rocksdb v7.4.3 2022-07-18T15:18:20+00:00 ## 7.4.3 (07/13/2022) ### Behavior Changes * For track_and_verify_wals_in_manifest, revert to the original behavior before #10087: syncing of live WAL file is not tracked, and we track only the synced sizes of **closed** WALs. (PR #10330). ## 7.4.2 (06/30/2022) ### Bug Fixes * Fix a bug in Logger where if dbname and db_log_dir are on different filesystems, dbname creation would fail wrt to db_log_dir path returning an error and fails to open the DB. ## 7.4.1 (06/28/2022) ### Bug Fixes * Pass `rate_limiter_priority` through filter block reader functions to `FileSystem`. ## 7.4.0 (06/19/2022) ### Bug Fixes * Fixed a bug in calculating key-value integrity protection for users of in-place memtable updates. In particular, the affected users would be those who configure `protection_bytes_per_key > 0` on `WriteBatch` or `WriteOptions`, and configure `inplace_callback != nullptr`. * Fixed a bug where a snapshot taken during SST file ingestion would be unstable. * Fixed a bug for non-TransactionDB with avoid_flush_during_recovery = true and TransactionDB where in case of crash, min_log_number_to_keep may not change on recovery and persisting a new MANIFEST with advanced log_numbers for some column families, results in "column family inconsistency" error on second recovery. As a solution, RocksDB will persist the new MANIFEST after successfully syncing the new WAL. If a future recovery starts from the new MANIFEST, then it means the new WAL is successfully synced. Due to the sentinel empty write batch at the beginning, kPointInTimeRecovery of WAL is guaranteed to go after this point. If future recovery starts from the old MANIFEST, it means the writing the new MANIFEST failed. We won't have the "SST ahead of WAL" error. * Fixed a bug where RocksDB DB::Open() may creates and writes to two new MANIFEST files even before recovery succeeds. Now writes to MANIFEST are persisted only after recovery is successful. * Fix a race condition in WAL size tracking which is caused by an unsafe iterator access after container is changed. * Fix unprotected concurrent accesses to `WritableFileWriter::filesize_` by `DB::SyncWAL()` and `DB::Put()` in two write queue mode. * Fix a bug in WAL tracking. Before this PR (#10087), calling `SyncWAL()` on the only WAL file of the db will not log the event in MANIFEST, thus allowing a subsequent `DB::Open` even if the WAL file is missing or corrupted. * Fix a bug that could return wrong results with `index_type=kHashSearch` and using `SetOptions` to change the `prefix_extractor`. * Fixed a bug in WAL tracking with wal_compression. WAL compression writes a kSetCompressionType record which is not associated with any sequence number. As result, WalManager::GetSortedWalsOfType() will skip these WALs and not return them to caller, e.g. Checkpoint, Backup, causing the operations to fail. * Avoid a crash if the IDENTITY file is accidentally truncated to empty. A new DB ID will be written and generated on Open. * Fixed a possible corruption for users of `manual_wal_flush` and/or `FlushWAL(true /* sync */)`, together with `track_and_verify_wals_in_manifest == true`. For those users, losing unsynced data (e.g., due to power loss) could make future DB opens fail with a `Status::Corruption` complaining about missing WAL data. * Fixed a bug in `WriteBatchInternal::Append()` where WAL termination point in write batch was not considered and the function appends an incorrect number of checksums. * Fixed a crash bug introduced in 7.3.0 affecting users of MultiGet with `kDataBlockBinaryAndHash`. * Add some fixes in async_io which was doing extra prefetching in shorter scans. ### Public API changes * Add new API GetUnixTime in Snapshot class which returns the unix time at which Snapshot is taken. * Add transaction `get_pinned` and `multi_get` to C API. * Add two-phase commit support to C API. * Add `rocksdb_transaction_get_writebatch_wi` and `rocksdb_transaction_rebuild_from_writebatch` to C API. * Add `rocksdb_options_get_blob_file_starting_level` and `rocksdb_options_set_blob_file_starting_level` to C API. * Add `blobFileStartingLevel` and `setBlobFileStartingLevel` to Java API. * Add SingleDelete for DB in C API * Add User Defined Timestamp in C API. * `rocksdb_comparator_with_ts_create` to create timestamp aware comparator * Put, Get, Delete, SingleDelete, MultiGet APIs has corresponding timestamp aware APIs with suffix `with_ts` * And Add C API's for Transaction, SstFileWriter, Compaction as mentioned [here](https://github.com/facebook/rocksdb/wiki/User-defined-Timestamp-(Experimental)) * The contract for implementations of Comparator::IsSameLengthImmediateSuccessor has been updated to work around a design bug in `auto_prefix_mode`. * The API documentation for `auto_prefix_mode` now notes some corner cases in which it returns different results than `total_order_seek`, due to design bugs that are not easily fixed. Users using built-in comparators and keys at least the size of a fixed prefix length are not affected. * Obsoleted the NUM_DATA_BLOCKS_READ_PER_LEVEL stat and introduced the NUM_LEVEL_READ_PER_MULTIGET and MULTIGET_COROUTINE_COUNT stats * Introduced `WriteOptions::protection_bytes_per_key`, which can be used to enable key-value integrity protection for live updates. ### New Features * Add FileSystem::ReadAsync API in io_tracing * Add blob garbage collection parameters `blob_garbage_collection_policy` and `blob_garbage_collection_age_cutoff` to both force-enable and force-disable GC, as well as selectively override age cutoff when using CompactRange. * Add an extra sanity check in `GetSortedWalFiles()` (also used by `GetLiveFilesStorageInfo()`, `BackupEngine`, and `Checkpoint`) to reduce risk of successfully created backup or checkpoint failing to open because of missing WAL file. * Add a new column family option `blob_file_starting_level` to enable writing blob files during flushes and compactions starting from the specified LSM tree level. * Add support for timestamped snapshots (#9879) * Provide support for AbortIO in posix to cancel submitted asynchronous requests using io_uring. * Add support for rate-limiting batched `MultiGet()` APIs ### Behavior changes * DB::Open(), DB::OpenAsSecondary() will fail if a Logger cannot be created (#9984) * Removed support for reading Bloom filters using obsolete block-based filter format. (Support for writing such filters was dropped in 7.0.) For good read performance on old DBs using these filters, a full compaction is required. * Per KV checksum in write batch is verified before a write batch is written to WAL to detect any corruption to the write batch (#10114). ### Performance Improvements * When compiled with folly (Meta-internal integration; experimental in open source build), improve the locking performance (CPU efficiency) of LRUCache by using folly DistributedMutex in place of standard mutex. 2022-07-18T15:18:20+00:00 rocksdb v7.4.4 rocksdb v7.4.4 2022-07-28T18:34:26+00:00 ## 7.4.4 (07/19/2022) ### Public API changes * Removed Customizable support for RateLimiter and removed its CreateFromString() and Type() functions. ### Bug Fixes * Fix a bug where `GenericRateLimiter` could revert the bandwidth set dynamically using `SetBytesPerSecond()` when a user configures a structure enclosing it, e.g., using `GetOptionsFromString()` to configure an `Options` that references an existing `RateLimiter` object. 2022-07-28T18:34:26+00:00 rocksdb v7.4.5 rocksdb v7.4.5 2022-08-02T23:17:54+00:00 ## 7.4.5 (08/02/2022) ### Bug Fixes * Fix a bug starting in 7.4.0 in which some fsync operations might be skipped in a DB after any DropColumnFamily on that DB, until it is re-opened. This can lead to data loss on power loss. (For custom FileSystem implementations, this could lead to `FSDirectory::Fsync` or `FSDirectory::Close` after the first `FSDirectory::Close`; Also, valgrind could report call to `close()` with `fd=-1`.) 2022-08-02T23:17:54+00:00 whids v1.8.0-beta.7 whids v1.8.0-beta.7 2022-08-03T12:33:11+00:00 2022-08-03T12:33:11+00:00 PyPCAPKit v0.16.2 PyPCAPKit v0.16.2 2022-08-04T02:55:47+00:00 2022-08-04T02:55:47+00:00 maltrail 0.48 maltrail 0.48 2022-08-04T06:36:06+00:00 Start-of-month release 2022-08-04T06:36:06+00:00 pcileech v4.15 pcileech v4.15 2022-08-04T06:39:53+00:00 * Support for MemProcFS v5.0 2022-08-04T06:39:53+00:00 MISP v2.4.160 MISP v2.4.160 2022-08-08T12:32:32+00:00 We are pleased to announce the immediate availability of MISP v2.4.160. With the August summer-holiday season kicking into high gear, we have a very special release for you all, containing a long list of major new features, improvements and general quality of life improvements. Unlike we do normally, this time around we're preparing separate blog posts for some of those major features, so follow the links below to read up on in-depth descriptions of each. # Workflows Something that has been in the works for quite some time now is finally hitting a release version of MISP, as of 2.4.160, we have the first release of the built in workflow system released. This system allows you to use an easy to use, yet extremely powerful graphical interface to modify how MISP handles certain tasks such as event publishing, user enrollment, synchronisation, etc., by adding additional logical steps in their respective executions, utilising a module system similar to what was already common to MISP from enrichment subsystems, exports as well as imports. This is merely the first step (or leap rather) towards customising and sharing custom workflows, stay tuned for new features, improvements as well as triggers and modules in the near future. Head over to the [README](https://github.com/MISP/misp-workflow-blueprints/blob/main/README.md) as well as a nifty [slide deck](https://www.misp-project.org/misp-training/a.12-misp-workflows.pdf), to find out what this incredibly powerful can do for you and your community. # New correlation engine One of the biggest pain points as of recently has been our dated and rather bloated correlation engine, which could easily bring a long running MISP instance to its knees when certain highly correlated data sources were synchronised. As of 2.4.160, we now have 2 brand new correlation engines at your disposal, with the old engine being retired immediately. Please be aware that upgrading to the current version will regenerate your correlations using the new engine, something that can take quite a long time (on our largest instance it took a whopping 40 hours!). With that said, we can assure you it's well worth the wait and should resolve several long standing performance bottlenecks as well as heavily cut down on the space requirements for your data. For more information, on the new engines, their differences, the various new support tools as well as what benefits you should expect, head over to the [dedicated blog post](https://github.com/MISP/MISP/blob/2.4/docs/correlations.rework.md). # STIX 2 library reworks There has been a massive amount of work going into the STIX 2.x library rework, bringing us closer and closer to having a full mapping of everything expressable. We're collaborating with CISA and Mitre to ensure that MISP can both express and understand STIX to its fullest extent. For more information, head over to the [release notes](https://github.com/MISP/misp-stix/releases/tag/v2.4.160) over on the MISP STIX library's repo. # Mermaid support for Event reports added Writing custom reports has become more and more popular, but one annoyance has been the lack of a way to depict graphs and flow charts without relying on external tools to create those (and share them as images for example). Using Mermaid, you now have a nifty tool to build graphs out of simple markdown directly in the event report editor. # Various other improvements A long list of other improvements, affecting the performance and stability of the platform as well as improvements to existing features. Head over to the changelog for a detailed list of changes. # Acknowledgement We would like to thank all the [contributors](https://www.misp-project.org/contributors), reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in [misp-objects](https://www.misp-project.org/objects.html), [misp-taxonomies](https://www.misp-project.org/taxonomies.html) and [misp-galaxy](https://www.misp-project.org/galaxy.html). As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. Additional changelogs are available for [misp galaxy](https://www.misp-project.org/Changelog-misp-galaxy.txt), [misp-taxonomies](https://www.misp-project.org/Changelog-misp-taxonomies.txt), [misp-objects](https://www.misp-project.org/Changelog-misp-objects.txt) and [misp-modules](https://www.misp-project.org/Changelog-misp-modules.txt) 2022-08-08T12:32:32+00:00 Lookyloo v1.14.0 Lookyloo v1.14.0 2022-08-08T13:53:34+00:00 # New features * Trigger a capture on a web enabled document provided as a file instead of a URL. Useful for HTML files attached to emails, or HTML body in email. ![Screenshot_20220808_131607](https://user-images.githubusercontent.com/248875/183406016-e02f99bd-fe8c-47ea-906e-39e9da2280b6.png) -------- * Compress (gzip) the HAR file in archived captures - saves a lot of disk space. * Support for RiskIQ Passive DNS (requires API key) * Display SSL/TLS information available in the HAR dump from Playwright ![Screenshot_20220808_132643](https://user-images.githubusercontent.com/248875/183407809-4475d6ce-2311-43fe-bb79-8a0697bae78c.png) -------- * Optional DoNotTrack HTTP header in capture ![Screenshot_20220808_132302](https://user-images.githubusercontent.com/248875/183407193-06aaf9a0-8377-49e7-bceb-5cb9dadad6fb.png) -------- * Display size of rendered page on hostnode popup. * [WiP] Download files when the URL captured to a downloadable file (PDF, Office doc, ...) (**Important note** the downloaded file is not exposed to the user yet) * [WiP] List all hashes available in the capture, sort them by frequency. Makes it easier to find phishing sides using the same resources. ![Screenshot_20220808_132149](https://user-images.githubusercontent.com/248875/183407015-daf83393-e605-4f3c-a1a2-0d3885023422.png) # Fixes * Major speed improvements when displaying the hostnode popups (only show the recent cached captures by default) * Improvements in the caching mechanism * Cleanup data showed by monitoring script * Avoid crashes when RiskIQ isn't reachable # Changes * Update dependencies (js, python) * Improve logging in archiver * Improve config file 2022-08-08T13:53:34+00:00 whids v1.8.0-beta.8 whids v1.8.0-beta.8 2022-08-08T16:38:56+00:00 2022-08-08T16:38:56+00:00 MISP v2.4.161 MISP v2.4.161 2022-08-11T15:30:58+00:00 We are pleased to announce the immediate availability of [MISP v2.4.161](https://github.com/MISP/MISP/releases/tag/v2.4.161). ![](https://www.misp-project.org//img/blog/workflow.png) # Small improvements - A new option added to log the last API request of an API key. (Thanks to Tom King for the contribution) - Overcorrelation features have some new improvements such as: - A new tool to generate occurrence counts (real numbers this time) - A hook to truncate the over-correlating value table on recorrelation - We no longer store the partial counts as occurrences when generating correlations - Performance improvements in event fetching - Various performance tuning in the new correlation engine including the full recorrelation # Bugs fixed - `tlp:amber+strict` and `tlp:clear` are now valid tags - [stix2 import] Better `external_references` parsing for attack patterns objects Thanks to all the contributors and users reporting bugs to make the software better. As always, a detailed and [complete changelog is available](https://www.misp-project.org/Changelog.txt) with all the fixes, changes and improvements in MISP core. 2022-08-11T15:30:58+00:00 osquery 5.5.0 osquery 5.5.0 2022-08-12T17:47:19+00:00 draft 2022-08-12T17:47:19+00:00 dnstwist 20220815 dnstwist 20220815 2022-08-15T15:39:49+00:00 2022-08-15T15:39:49+00:00