http://open-source-security-software.net/releases.atom Recent releases 2021-04-18T21:42:14.851288+00:00 python-feedgen traefik v1.7.29 traefik v1.7.29 2021-03-29T16:55:10+00:00 - Update to go1.16 ([#7967](https://github.com/traefik/traefik/pull/7967) by [ldez](https://github.com/ldez)) 2021-03-29T16:55:10+00:00 TheHive 4.1.2 TheHive 4.1.2 2021-03-29T17:05:02+00:00 **Implemented enhancements:** - [Feature Request] Add case search by TTP [\#1893](https://github.com/TheHive-Project/TheHive/issues/1893) **Fixed bugs:** - [Bug] Slow loading of TheHive because of Tags [\#1869](https://github.com/TheHive-Project/TheHive/issues/1869) - [Bug] After migration from 4.0.5 to 4.1.0 old tasklogs are not returned by "/api/v1/query?name=case-task-logs" query [\#1875](https://github.com/TheHive-Project/TheHive/issues/1875) - Dashboards - custom fields [\#1877](https://github.com/TheHive-Project/TheHive/issues/1877) - [Bug] TH 4.1.1 : Filter by "IMPORTED" does not work for alerts imported into existing cases [\#1891](https://github.com/TheHive-Project/TheHive/issues/1891) - [Bug] Fix the S3 configuration options [\#1892](https://github.com/TheHive-Project/TheHive/issues/1892) - [Bug] All attachments in task logs disappeared following upgrade to 4.1.1 [\#1894](https://github.com/TheHive-Project/TheHive/issues/1894) - [Bug] Continued performance issues after upgrade to 4.1.1 [\#1896](https://github.com/TheHive-Project/TheHive/issues/1896) - [Bug] Fix issues dashboard list [\#1901](https://github.com/TheHive-Project/TheHive/issues/1901) - [Bug] Migration tool migrates unsupported elastic index [\#1907](https://github.com/TheHive-Project/TheHive/issues/1907) - [Bug] Folder permissions are not correctly set in docker image [\#1908](https://github.com/TheHive-Project/TheHive/issues/1908) 2021-03-29T17:05:02+00:00 flanker v0.9.14 flanker v0.9.14 2021-03-29T19:21:53+00:00 Encode Sender as an address header to avoid improper encoding of email addresses. 2021-03-29T19:21:53+00:00 PyMOSP v0.4.0 PyMOSP v0.4.0 2021-03-30T22:22:36+00:00 New ~~~ - [command line] It is now possible to use PyMOSP as a command line tool. Only for listing objects for the moment. [Cédric Bonhomme] - [objects] Added creation of objects and some tests. [Cédric Bonhomme] - [tests] added files for the future tests on JSON schemas. [Cédric Bonhomme] Changes ~~~~~~~ - [release] Bumped version number. [Cédric Bonhomme] - [tests] Lint with Flake8. [Cédric Bonhomme] - [tests] added test on get objects with uuid and language. [Cédric Bonhomme] - Updated .gitgignore to ignore .python-version file. [Cédric Bonhomme] Fix ~~~ - Typo. [Cédric Bonhomme] - [documentation] Fixed example file. [Cédric Bonhomme] - [type] Fixed type of result of objects() and add_objects() function. [Cédric Bonhomme] Other ~~~~~ - Updated chagenlog. [Cédric Bonhomme] - Updated chagenlog. [Cédric Bonhomme] - Updated README and pyproject.toml. [Cédric Bonhomme] - Fixed bad format of link in README. [Cédric Bonhomme] - Replaced str by map when joining Python version number. [Cédric Bonhomme] - Removed useless file. [Cédric Bonhomme] 2021-03-30T22:22:36+00:00 PyMOSP v0.4.1 PyMOSP v0.4.1 2021-03-30T22:23:00+00:00 Fix ~~~ - [documentation] fix the documentation in the README. #HowToFuckUpARelease. [Cédric Bonhomme] Other ~~~~~ - Updated README. [Cédric Bonhomme] 2021-03-30T22:23:00+00:00 rocksdb v6.16.4 rocksdb v6.16.4 2021-03-30T22:59:25+00:00 ## 6.16.4 (2021-03-30) ### Bug Fixes * Fix build on ppc64 and musl build. 2021-03-30T22:59:25+00:00 turbinia 20210330 turbinia 20210330 2021-03-31T00:00:29+00:00 2021-03-31T00:00:29+00:00 faraday v3.14.3 faraday v3.14.3 2021-03-31T05:16:07+00:00 3.14.3 [Mar 30th, 2021]: --- * MOD MAYOR Breaking change: Use frontend from other repository * ADD `last_run` to executors and agents * ADD ignore info vulns option (from faraday-plugins 1.4.3) * ADD invalid logins are registered in `audit.log` * ADD agent registration tokens are now 6-digit short and automatically regenerated every 30 seconds * MOD Fix logout redirect loop * REMOVE support for native SSL 2021-03-31T05:16:07+00:00 netdata v1.30.0 netdata v1.30.0 2021-03-31T12:30:55+00:00 The v1.30.0 release of Netdata brings major improvements to our packaging and completely replaces Google Analytics/GTM for product telemetry. We're also releasing the first changes in an upcoming overhaul to both our dashboard UI/UX and the suite of preconfigured alarms that comes with every installation. v1.30.0 contains 3 new collectors, 3 enhancements to notifications method, 38 improvements (13 in the dashboard), 16 documentation updates, and 17 bug fixes. ## At a glance The **ACLK-NG** is a much faster method of securely connecting a node to Netdata Cloud. In addition, there are no external dependencies to our custom [libmosquitto](https://github.com/netdata/mosquitto) and [libwebsockets](https://github.com/warmcat/libwebsockets) libraries, which means there's no more need to build these during installation. To enable ACLK-NG on a node that's already running the Netdata Agent, reinstall with the `--aclk-ng` option: ```bash bash <(curl -Ss https://my-netdata.io/kickstart.sh) --aclk-ng --reinstall ``` We **replaced Google Analytics/GTM**, which we used for collecting product telemetry, with a self-hosted instance of the open-source [PostHog](https://posthog.com/) project. When sending statistics to PostHog, any fields that might contain identifiable information, such as an IP address or URL, are hardcoded. If you previously opted-out of anonymous statistics, this migration does not change your existing settings. We also published a **developer environment** (devenv) to simplify contributing to the Netdata Agent. The devenv packages everything you need to develop improvements on the Netdata Agent itself, or its collectors, in a single Docker image. Read more about this devenv, and get started, in the [Netdata community repo](https://github.com/netdata/community/tree/main/devenv). ## Acknowledgments - [@aazedo](https://github.com/aazedo) for adding collection of attribute 233 (Media Wearout Indicator (SSD)) to the smartd_log collector - [@ossimantylahti](https://github.com/ossimantylahti) for fixing a typo in the email notifications readme - [@KickerTom](https://github.com/KickerTom) for renaming abs to ABS to avoid clash with standard definitions - [@Steve8291](https://github.com/Steve8291) for improving email, cron and ups groups in the apps_group.conf - [@liepumartins](https://github.com/liepumartins) for adding wireguard to the vpn group in the apps_group.conf - [@eltociear](https://github.com/eltociear) for fixing typos in main.h, backend_prometheus.c and dashboard_info.js - [@Habetdin](https://github.com/Habetdin) for fixing broken external links in the WEB GUI - [@salazarp](https://github.com/salazarp) for updating the syntax for Caddy v2 - [@RaitoBezarius](https://github.com/RaitoBezarius) for adding support to change IRC_PORT ## Improvements - Support VS Code container devenv. ([#10723](https://github.com/netdata/netdata/pull/10723), [@OdysLam](https://github.com/OdysLam)) - Add check for children connecting to a parent agent with an unsupported memory mode. ([#10787](https://github.com/netdata/netdata/pull/10787), [@stelfrag](https://github.com/stelfrag)) - Add lock check to avoid shutdown when compiled with internal and locking checks. ([#10835](https://github.com/netdata/netdata/pull/10835), [@stelfrag](https://github.com/stelfrag)) - Update chart's metadata in database when it already exists during creation. ([#10728](https://github.com/netdata/netdata/pull/10728), [@stelfrag](https://github.com/stelfrag)) - ACLK separate HTTPS client. ([#10784](https://github.com/netdata/netdata/pull/10784), [@underhood](https://github.com/underhood)) - Add new ACLK implementation (`ACLK-NG`). ([#10315](https://github.com/netdata/netdata/pull/10315), [@underhood](https://github.com/underhood)) - Add CPU statistics per ALCK query thread. ([#10634](https://github.com/netdata/netdata/pull/10634), [@MrZammler](https://github.com/MrZammler)) - Add `_aclk_impl` label to the `/api/v1/info` endpoint. ([#10778](https://github.com/netdata/netdata/pull/10778), [@underhood](https://github.com/underhood)) - Add a new `chart` parameter to the `/api/v1/alarm_log` endpoint. ([#10788](https://github.com/netdata/netdata/pull/10788), [@MrZammler](https://github.com/MrZammler)) - Add data query support for archived charts. ([#10771](https://github.com/netdata/netdata/pull/10771), [@stelfrag](https://github.com/stelfrag)) - Add HTTP cookie (SameSite, Secure). ([#10676](https://github.com/netdata/netdata/pull/10676), [@thiagoftsm](https://github.com/thiagoftsm)) - Add statistics per Cloud query type. ([#10602](https://github.com/netdata/netdata/pull/10602), [@underhood](https://github.com/underhood)) - Add support for changing the number of pages per database engine extent. ([#10593](https://github.com/netdata/netdata/pull/10593), [@mfundul](https://github.com/mfundul)) - Add the ability to store chart labels in the database. ([#10718](https://github.com/netdata/netdata/pull/10718), [@stelfrag](https://github.com/stelfrag)) - Enable metadata persistence in all memory modes. ([#10742](https://github.com/netdata/netdata/pull/10742), [@stelfrag](https://github.com/stelfrag)) - Increase `curl connect-timeout` and decrease number of claim attempts. ([#10800](https://github.com/netdata/netdata/pull/10800), [@ilyam8](https://github.com/ilyam8)) - Increase the ACLK exponential backoff randomness. ([#10373](https://github.com/netdata/netdata/pull/10373), [@underhood](https://github.com/underhood)) - Log ACLK Cloud commands to `access.log`. ([#10697](https://github.com/netdata/netdata/pull/10697), [@stelfrag](https://github.com/stelfrag)) - Remove an unused function warning in legacy version of the ACLK. ([#10731](https://github.com/netdata/netdata/pull/10731), [@underhood](https://github.com/underhood)) - Remove unreachable #else directives in plugins. ([#10523](https://github.com/netdata/netdata/pull/10523), [@vkalintiris](https://github.com/vkalintiris)) - Rename `struct avl` to `avl_element` and the `typedef` to `avl_t`. ([#10735](https://github.com/netdata/netdata/pull/10735), [@vkalintiris](https://github.com/vkalintiris)) - Replace Google Analytics with PostHog for backend telemetry events. ([#10636](https://github.com/netdata/netdata/pull/10636), [@andrewm4894](https://github.com/andrewm4894)) - Skip C++ incompatible header in main libnetdata header. ([#10737](https://github.com/netdata/netdata/pull/10737), [@vkalintiris](https://github.com/vkalintiris)) - Try to keep all pages from extents read from disk in the cache. ([#10558](https://github.com/netdata/netdata/pull/10558), [@mfundul](https://github.com/mfundul)) - Use a parameter name that is not a reserved keyword in C++. ([#10738](https://github.com/netdata/netdata/pull/10738), [@vkalintiris](https://github.com/vkalintiris)) - Use of out-of-line struct definitions. ([#10739](https://github.com/netdata/netdata/pull/10739), [@vkalintiris](https://github.com/vkalintiris)) ## Dashboard - Add `max` value to the `nvidia_smi.fan_speed` gauge. ([#10780](https://github.com/netdata/netdata/pull/10780), [@ilyam8](https://github.com/ilyam8)) - Add state map to duplex and operstate charts. ([#10752](https://github.com/netdata/netdata/pull/10752), [@vlvkobal](https://github.com/vlvkobal)) - Add supervisord to `dashboard_info.js`. ([#10754](https://github.com/netdata/netdata/pull/10754), [@ilyam8](https://github.com/ilyam8)) - Fix broken external links. ([#10586](https://github.com/netdata/netdata/pull/10586), [@Habetdin](https://github.com/Habetdin)) - Make network state map syntax consistent in `dashboard_info.js`. ([#10849](https://github.com/netdata/netdata/pull/10849), [@ilyam8](https://github.com/ilyam8)) - dashboard@v2.13.28 ([#10761](https://github.com/netdata/netdata/pull/10761), [@jacekkolasa](https://github.com/jacekkolasa)) - Fix alarms log export. - Persist relative timeframe. - Allow multirow names in the replicated nodes list. - Fix the date & time picker overlap. - Update Font Awesome. - Truncate long names. - Update links: change `docs.netdata.cloud` to l`earn.netdata.cloud`. - Remove Google's GA & GTM completely, in favor of open-source PostHog. ## Health ### Bug fixes - Fix delaying CLEAR notifications when using the `repeat` feature. ([#10846](https://github.com/netdata/netdata/pull/10846), [@thiagoftsm](https://github.com/thiagoftsm)) - Fix wrong count of entries in the `alarm.log`. ([#10564](https://github.com/netdata/netdata/pull/10564), [@thiagoftsm](https://github.com/thiagoftsm)) ### Alarms - Add `wmi_` prefix to the wmi collector network alarms. ([#10782](https://github.com/netdata/netdata/pull/10782), [@ilyam8](https://github.com/ilyam8)) - Add collector prefix to the external collectors alarms. ([#10830](https://github.com/netdata/netdata/pull/10830), [@ilyam8](https://github.com/ilyam8)) - Apply adapter_raid alarms for every logical/physical device. ([#10820](https://github.com/netdata/netdata/pull/10820), [@ilyam8](https://github.com/ilyam8)) - Apply megacli alarms for every adapter/physical disk. ([#10834](https://github.com/netdata/netdata/pull/10834), [@ilyam8](https://github.com/ilyam8)) - Exclude cgroups network interfaces from packets dropped alarms. ([#10806](https://github.com/netdata/netdata/pull/10806), [@ilyam8](https://github.com/ilyam8)) - Fix various alarms critical and warning thresholds hysteresis. ([#10779](https://github.com/netdata/netdata/pull/10779), [@ilyam8](https://github.com/ilyam8)) - Improve alarms `info` fields. ([#10853](https://github.com/netdata/netdata/pull/10853), [@ilyam8](https://github.com/ilyam8)) - Make VerneMQ alarms less sensitive. ([#10770](https://github.com/netdata/netdata/pull/10770), [@ilyam8](https://github.com/ilyam8)) - Make alarms less sensitive. ([#10688](https://github.com/netdata/netdata/pull/10688), [@ilyam8](https://github.com/ilyam8)) - Remove `exporting_metrics_lost` template. ([#10829](https://github.com/netdata/netdata/pull/10829), [@ilyam8](https://github.com/ilyam8)) - Remove `ram_in_swap` alarm. ([#10789](https://github.com/netdata/netdata/pull/10789), [@ilyam8](https://github.com/ilyam8)) - Use separate `packets_dropped_ratio` alarms for wireless network interfaces. ([#10785](https://github.com/netdata/netdata/pull/10785), [@ilyam8](https://github.com/ilyam8)) ### Notifications - Add ability to change port number when using IRC notification method. ([#10824](https://github.com/netdata/netdata/pull/10824), [@RaitoBezarius](https://github.com/RaitoBezarius)) - Add `dump_methods` parameter to `alarm-notify.sh.in`. ([#10772](https://github.com/netdata/netdata/pull/10772), [@MrZammler](https://github.com/MrZammler)) - Log an error if there is a failure during an email alarm notification. ([#10818](https://github.com/netdata/netdata/pull/10818), [@ilyam8](https://github.com/ilyam8)) ## Collectors ### New - Add monitoring of synchronization system calls to the eBPF collector. ([#10814](https://github.com/netdata/netdata/pull/10814), [@thiagoftsm](https://github.com/thiagoftsm)) - Add monitoring of Linux page cache to the eBPF collector. ([#10693](https://github.com/netdata/netdata/pull/10693), [@thiagoftsm](https://github.com/thiagoftsm)) ### Improvements - Add `k6.conf` to the StatsD collector. ([#10733](https://github.com/netdata/netdata/pull/10733), [@OdysLam](https://github.com/OdysLam)) - Clean up the eBPF collector. ([#10680](https://github.com/netdata/netdata/pull/10680), [@thiagoftsm](https://github.com/thiagoftsm)) - Use working set for memory utilization in the cgroups collector. ([#10712](https://github.com/netdata/netdata/pull/10712), [@vlvkobal](https://github.com/vlvkobal)) - Add new configuration parameters to the example Python collector. ([#10777](https://github.com/netdata/netdata/pull/10777), [@andrewm4894](https://github.com/andrewm4894)) - Add carrier and MTU charts for network interfaces. ([#10866](https://github.com/netdata/netdata/pull/10866), [@vlvkobal](https://github.com/vlvkobal)) - Improve email, cron, and UPS groups in the `apps.plugin` configuration. ([#9313](https://github.com/netdata/netdata/pull/9313), [@Steve8291](https://github.com/Steve8291)) - Add Wireguard to the `vpn` group in the `apps.plugin` configuration. ([#10743](https://github.com/netdata/netdata/pull/10743), [@liepumartins](https://github.com/liepumartins)) - Add alarm values collection to the Python alarms collector. ([#10675](https://github.com/netdata/netdata/pull/10675), [@andrewm4894](https://github.com/andrewm4894)) - Add `attribute 233` (Media Wearout Indicator (SSD)) collection to the python smartd_log collector. ([#10711](https://github.com/netdata/netdata/pull/10711), [@aazedo](https://github.com/aazedo)) - Move network interface speed, duplex, and operstate variables to charts. ([#10740](https://github.com/netdata/netdata/pull/10740), [@vlvkobal](https://github.com/vlvkobal)) - Update `go.d.plugin` version to v0.28.1. ([#10826](https://github.com/netdata/netdata/pull/10826), [@ilyam8](https://github.com/ilyam8)) - Add a `noauthcodecheck` workaround flag to the freeipmi collector. ([#10701](https://github.com/netdata/netdata/pull/10701), [@vlvkobal](https://github.com/vlvkobal)) ### Bug fixes - Fix eBPF collector compatibility with kernels v5.11+. ([#10707](https://github.com/netdata/netdata/pull/10707), [@thiagoftsm](https://github.com/thiagoftsm)) - Fix disks identification in the diskstats collector. ([#10843](https://github.com/netdata/netdata/pull/10843), [@vlvkobal](https://github.com/vlvkobal)) - Fix the count of `cpuset.cpus` in the cgroups collector. ([#10757](https://github.com/netdata/netdata/pull/10757), [@ilyam8](https://github.com/ilyam8)) - Fix disk utilization and backlog charts in the diskstats collector. ([#10705](https://github.com/netdata/netdata/pull/10705), [@vlvkobal](https://github.com/vlvkobal)) ## Exporting ### Bug fixes - Fix adding `duplicate _total` suffixes for the Prometheus collector. ([#10674](https://github.com/netdata/netdata/pull/10674), [@vlvkobal](https://github.com/vlvkobal)) ## Packaging and installation - Add JSON output option for `buildinfo`. ([#10706](https://github.com/netdata/netdata/pull/10706), [@Ferroin](https://github.com/Ferroin)) - Add information about the `--aclk-ng` option to the netdata-installer script. ([#10852](https://github.com/netdata/netdata/pull/10852), [@underhood](https://github.com/underhood)) - Add support for claiming nodes as part of installation. ([#10084](https://github.com/netdata/netdata/pull/10084), [@Ferroin](https://github.com/Ferroin)) - Assorted updater fixes ([#10613](https://github.com/netdata/netdata/pull/10613), [@Ferroin](https://github.com/Ferroin)) - Fix claiming via environment variables in a Docker container ([#10811](https://github.com/netdata/netdata/pull/10811), [@ilyam8](https://github.com/ilyam8)) - Fix detection of already claimed node in Docker images ([#10720](https://github.com/netdata/netdata/pull/10720), [@Ferroin](https://github.com/Ferroin)) - Fix handling of perf.plugin capabilities ([#10766](https://github.com/netdata/netdata/pull/10766), [@Ferroin](https://github.com/Ferroin)) - Fix handling of permissions for some plugins ([#10490](https://github.com/netdata/netdata/pull/10490), [@Ferroin](https://github.com/Ferroin)) ## Documentation - Add guide: _Develop a custom data collector for Netdata in Python_. ([#10710](https://github.com/netdata/netdata/pull/10710), [@joelhans](https://github.com/joelhans)) - Add guide: _LAMP stack monitoring_. ([#10698](https://github.com/netdata/netdata/pull/10698), [@joelhans](https://github.com/joelhans)) - Add guide: _Unsupervised anomaly detection for Raspberry Pi monitoring_. ([#10713](https://github.com/netdata/netdata/pull/10713), [@joelhans](https://github.com/joelhans)) - Add guide: _How to use any StatsD data source with Netdata_. ([#10719](https://github.com/netdata/netdata/pull/10719), [@OdysLam](https://github.com/OdysLam)) - Convert references to `service` to `systemctl`. ([#10703](https://github.com/netdata/netdata/pull/10703), [@joelhans](https://github.com/joelhans)) - Fix broken link in StatsD guide. ([#10831](https://github.com/netdata/netdata/pull/10831), [@joelhans](https://github.com/joelhans)) - Fix broken links in active alarms doc. ([#10678](https://github.com/netdata/netdata/pull/10678), [@joelhans](https://github.com/joelhans)) - Improve the Kubernetes deployment documentation. ([#10662](https://github.com/netdata/netdata/pull/10662), [@joelhans](https://github.com/joelhans)) - Revamp StatsD docs. ([#10637](https://github.com/netdata/netdata/pull/10637), [@OdysLam](https://github.com/OdysLam)) - Update guide: _Kubernetes monitoring with Netdata: Overview and visualizations_. ([#10691](https://github.com/netdata/netdata/pull/10691), [@joelhans](https://github.com/joelhans)) - Update screenshots and text for new Cloud navigation. ([#10664](https://github.com/netdata/netdata/pull/10664), [@joelhans](https://github.com/joelhans)) - Comment out `memory mode` mention in StatsD example. ([#10751](https://github.com/netdata/netdata/pull/10751), [@OdysLam](https://github.com/OdysLam)) - Fix a typo in the email notifications doc. ([#10668](https://github.com/netdata/netdata/pull/10668), [@ossimantylahti](https://github.com/ossimantylahti)) - Update syntax for Caddy v2. ([#10823](https://github.com/netdata/netdata/pull/10823), [@salazarp](https://github.com/salazarp)) ## Bug fixes - Fix a typo in `main.h`. ([#10858](https://github.com/netdata/netdata/pull/10858), [@eltociear](https://github.com/eltociear)) - Fix a typo in `backend_prometheus.c`. ([#10716](https://github.com/netdata/netdata/pull/10716), [@eltociear](https://github.com/eltociear)) - Fix a typo in `dashboard_info.js`. ([#10775](https://github.com/netdata/netdata/pull/10775), [@eltociear](https://github.com/eltociear)) - Fix segfault due to misalignment between global and StatsD memory modes. ([#10732](https://github.com/netdata/netdata/pull/10732), [@stelfrag](https://github.com/stelfrag)) - Fix zombie alarms for charts that are obsolete/removed. ([#10804](https://github.com/netdata/netdata/pull/10804), [@vlvkobal](https://github.com/vlvkobal)) - Fix a Coverity warning in the new MQTT library. ([#10851](https://github.com/netdata/netdata/pull/10851), [@underhood](https://github.com/underhood)) - Fix a parameter binding issue when storing chart names in the database. ([#10717](https://github.com/netdata/netdata/pull/10717), [@stelfrag](https://github.com/stelfrag)) - Fix crash when executing data query with context and non-existing `chart_label_key`. ([#10844](https://github.com/netdata/netdata/pull/10844), [@stelfrag](https://github.com/stelfrag)) - Fix claiming behind Squid proxy. ([#10734](https://github.com/netdata/netdata/pull/10734), [@underhood](https://github.com/underhood)) - Fix Coverity issue (CID 367566). ([#10813](https://github.com/netdata/netdata/pull/10813), [@stelfrag](https://github.com/stelfrag)) - Fix memory leak when archived data is requested. ([#10837](https://github.com/netdata/netdata/pull/10837), [@stelfrag](https://github.com/stelfrag)) - Fix clash with C++ standard definitions by changing `abs` to `ABS`. ([#10354](https://github.com/netdata/netdata/pull/10354), [@KickerTom](https://github.com/KickerTom)) 2021-03-31T12:30:55+00:00 AIL-framework v3.5 AIL-framework v3.5 2021-03-31T13:24:17+00:00 # AIL Framework version 3.5 released with a new flexible crawler manager, built-in Passive DNS sensor and many fixes. ![ail-email-corr](https://user-images.githubusercontent.com/3309/113150939-e3a30b00-9234-11eb-8fe6-912d0ec3b8d8.png) ![ail-key-leak](https://user-images.githubusercontent.com/3309/113151048-003f4300-9235-11eb-8aa1-966493dfadf7.png) ![ail-crawler-manager](https://user-images.githubusercontent.com/3309/113150987-f3225400-9234-11eb-8999-28248667a246.png) # Changes - [passivedns] D4server port. [Terrtia] - [v3.5] install pyAIL. [Terrtia] - [passivedns] D4server port. [Terrtia] - [passiveDns] add dns records. [Terrtia] - [passiveDns D4 Client] add passiveDns D4 Client. [Terrtia] - [crawler] add test + relaunch crawlers + major fixs. [Terrtia] - [update] add v3.5 update. [Terrtia] - [doc] Splash Manager Configuration. [Terrtia] - [UI] add update note. [Terrtia] - [IPAddress] Add Debug + Check if option is empty. [Terrtia] - [splash manager] update enpoints + use Splash name to restart docker. [Terrtia] - [merge master] [Terrtia] - [domains search] search domains by name. [Terrtia] - [screenshot + har directory] add option to change screenshots directory. [Terrtia] - [crawler_manager] UI edit config + fix crawler queues. [Terrtia] - [config] save config on DB + default values. [Terrtia] - [crawlers manager] show setings. [Terrtia] - [crawler] show all crawlers type on dashboard. [Terrtia] - [crawler] manage crawlers. [Terrtia] - [crawler] crawler queue + restart docker on error. [Terrtia] - [core module] disable phone module by default. [Terrtia] - Merge master -> crawler_manager. [Terrtia] - [crawler manager] get all splash dockers, proxies and launch all crawlers. [Terrtia] # Fixes - [crawler] crawler test: remove print. [Terrtia] - [crawler UI] remove unused crawlers configuration. [Terrtia] - [crawler] user agent + splash restart. [Terrtia] - [crawler UI] crawler by domain type, remove old dashboard. [Terrtia] - [MISP Export UI] object type selector. [Terrtia] - [UI] fix navbar: invalid url. [Terrtia] - [crawler] typo. [Terrtia] - [crawler] typo. [Terrtia] - [crawler] typo. [Terrtia] - [Crawler] faup. [Terrtia] - [Launcher] launch core module: Crawler_manager. [Terrtia] - [cralers] remove debug. [Terrtia] - [Domain search] fix regex. [Terrtia] - [API ACL] avoid user_no_api users to access the API. [Terrtia] - [Module Queue] module without subscriber. [Terrtia] - [term Tracker] TimeoutException. [Terrtia] - [API ACL] read_only user role. [Terrtia] - [OVERVIEW] [Terrtia] - [gitignore] [Terrtia] - [showDomain] empty father field. [Terrtia] - [showDomain] empty father field. [Terrtia] - [redis cache] remove old Redis_Data_Merging db. [Terrtia] - [redis cache] remove old paste_name db. [Terrtia] - [background updater] Don't launch updates on fresh install. [Terrtia] - [v3.4 update] fix update progress. [Terrtia] - [background updater] remove completed updates. [Terrtia] - [install] force virtual environmemt activation. [Terrtia] # Other - Merge branch 'master' into crawler_manager. [Terrtia] - Merge pull request #89 from ail-project/crawler_manager. [Alexandre Dulaunoy] Crawler manager - Merge branch 'master' into crawler_manager. [Terrtia] - Merge branch 'master' into crawler_manager. [Terrtia] - Merge pull request #536 from simonsigre/patch-1. [Alexandre Dulaunoy] Typo in placeholder "Optionnal" --> "Optional" - Typo in placeholder "Optionnal" --> "Optional" [Simon Sigré] Updated placeholder value to correct typo; "Optionnal" --> "Optional" - Merge branch 'master' of github.com:CIRCL/AIL-framework. [Alexandre Dulaunoy] - Merge pull request #534 from simonsigre/patch-1. [Alexandre Dulaunoy] Added 'wget' as a dependency - Added 'wget' as a dependency. [Simon Sigré] 'wget' is a dependency as without this additional components won't download as part of initial install, one such example is; ``` wget -O /ail-framework/ardb/src/../deps/jemalloc-5.1.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/5.1.0/jemalloc-5.1.0.tar.bz2 && \ ``` - Merge branch 'master' into crawler_manager. [Terrtia] - Merge branch 'master' into crawler_manager. [Terrtia] - Merge branch 'master' into crawler_manager. [Terrtia] 2021-03-31T13:24:17+00:00 PyMOSP v0.4.2 PyMOSP v0.4.2 2021-03-31T13:42:10+00:00 Changes ~~~~~~~ - [core] Python requirement set to >=3.8,<4.0. [Cédric Bonhomme] 2021-03-31T13:42:10+00:00 pharos ghidra-9.1 pharos ghidra-9.1 2021-03-31T13:47:57+00:00 2021-03-31T13:47:57+00:00 pharos ghidra-9.0.4 pharos ghidra-9.0.4 2021-03-31T13:48:18+00:00 2021-03-31T13:48:18+00:00 pharos ghidra-9.1.1 pharos ghidra-9.1.1 2021-03-31T13:48:28+00:00 2021-03-31T13:48:28+00:00 pharos ghidra-9.2 pharos ghidra-9.2 2021-03-31T13:48:32+00:00 2021-03-31T13:48:32+00:00 pharos ghidra-9.1.2 pharos ghidra-9.1.2 2021-03-31T13:48:34+00:00 2021-03-31T13:48:34+00:00 pharos ghidra-9.2.2 pharos ghidra-9.2.2 2021-03-31T13:48:47+00:00 2021-03-31T13:48:47+00:00 pharos ghidra-9.2.1 pharos ghidra-9.2.1 2021-03-31T13:49:11+00:00 2021-03-31T13:49:11+00:00 pharos ghidra-master pharos ghidra-master 2021-03-31T13:59:51+00:00 2021-03-31T13:59:51+00:00 binskim v1.7.5-prerelease1 binskim v1.7.5-prerelease1 2021-03-31T13:59:53+00:00 ## **v1.7.5-prerelease1** [NuGet Package](https://www.nuget.org/packages/Microsoft.CodeAnalysis.BinSkim/1.7.5-prerelease1) * BUGFIX: Fix import/export config using JSON file. [#349](https://github.com/microsoft/binskim/pull/349) * FEATURE: Add compiler report rule BA4001, which is disabled by default. [#350](https://github.com/microsoft/binskim/pull/350) * FEATURE: Add support to specific rule documentation in `HelpUri`. [#348](https://github.com/microsoft/binskim/pull/348) 2021-03-31T13:59:53+00:00 mattermost-server v5.33.3 mattermost-server v5.33.3 2021-03-31T16:24:27+00:00 Mattermost Platform Release v5.33.3 2021-03-31T16:24:27+00:00 node_redis v3.0.2 node_redis v3.0.2 2021-03-31T21:28:49+00:00 2021-03-31T21:28:49+00:00 node_redis v3.1.0 node_redis v3.1.0 2021-03-31T21:55:53+00:00 # Enhancements * Upgrade node and dependencies and redis-commands to support Redis 6 (#1578) * Add support for Redis 6 `auth pass [user]` (#1508) 2021-03-31T21:55:53+00:00 maltrail 0.31 maltrail 0.31 2021-03-31T23:01:08+00:00 Start-of-month release 2021-03-31T23:01:08+00:00 chipsec 1.6.0 chipsec 1.6.0 2021-04-01T03:45:01+00:00 __New or Updated Modules/UtilCmds:__ * `common.memlock` - Add display of register when in verbose mode * `common.smm_code_chk` - Capture self.res and fix encoding issue * `common.spi_access` - Add messaging for SPI checks * `common.spi_desc` - Add messaging for SPI checks * `utilcmd.config_cmd` - Add util config util command * `utilcmd.interrupts_cmd` - Update interrupts_cmd.py CLI to argparse and add smmc port option and print textual representation of the status returned by send_smmc_SMI * `utilcmd.io_cmd` - Fix parameter processing in I/O command * `utilcmd.pci_cmd` - Update pci_cmd.py CLI to argparse __New or Updated Configurations:__ * `8086/apl` - Updated the product xml files to match the datasheets * `8086/cht` - Updated the product xml files to match the datasheets * `8086/cml` - Add CML configuration * `8086/glk` - Updated the product xml files to match the datasheets * `8086/pch_4xx` - Add pch4xx config file * `8086/pch_4xxh` - Add pch4xxh config file * `8086/pch_5xxh` - Add pch5xxh config file * `8086/pch_5xxlp` - Add pch5xxlp config file * `8086/rkl` - Add RKL configuration * `8086/skx` - Add CPUID range to include detection_value for CPX * `8086/tglu` - Add TGLU configuration __Removed Modules:__ * None __Additional Changes:__ * `README.md` - Add release convention section * `chipsec.chipset` - Add execeptions within cs.read_register, specify detection_value ranges and check if the VID is present in the chipset dictionary * `chipsec/modules/tools/uefi/blockedlist.json` - Add MosaicRegressor UEFI RK to blockedlist * `chipsec_tools/log_parser/requirements.txt` - Update log_parser requirements.txt and add lxml version update * `drivers/linux/Makefile` - Support overriding Linux source path * `drivers/linux/dkms.conf` - Update chipsec version in dkms.conf and override KSRC instead of KERNELDIR in dkms.conf * `drivers/win7/chipsec_hlpr.vcxproj` - Fix copy error when building driver with a space in the path * `drivers/win7/driver.c` - Change macro used for writing to MMIO * `drivers/win7/include/chipsec.h` - Add IOCTL code * `hal.interrupts` - Add port options to send_smmc_SMI() and dump the communication buffer before and after triggering the SMI * `hal.uefi_common` - Dump boot script entries using dump_buffer_bytes and add textual representation of the status returned by send_smmc_SMI * `hal.uefi_platform` - Fix wrong header length for EDK-type boot script * `helper.linux.linuxhelper` - Fix mmio access granularity in native mode and error message when the Linux kernel module fails to load * `helper.win.win32helper` - Use new IOCTL for MMIO writes * `Sphinx` - Updates for manual * `tests.software.mock_helper` - Fix mock_helper for cs_exceptions * `travis.yml` - Update travis.yml for macos __Additional Notes:__ * __For Windows users, please rebuild your windows driver as updates have been made.__ * __For Windows, Linux and MacOS: Python2 support has been deprecated as of June 2020. Please use Python3.__ * Any modules under the `modules.tools` directory have not yet been fully validated to work with Python3. __New Module Details:__ Module Name | Supported Platforms | Flags :---: | :---: | :---: 2021-04-01T03:45:01+00:00 pm2 4.5.6 pm2 4.5.6 2021-04-01T10:20:19+00:00 - all submodules version bumped - supress all "security" warnings 2021-04-01T10:20:19+00:00 scancode-toolkit v21.3.31 scancode-toolkit v21.3.31 2021-04-01T15:21:49+00:00 This is a major version with no breaking API changes. Attention: the next version will bring up some significant API changes summarized in the [CHANGELOG](https://github.com/nexB/scancode-toolkit/blob/fa3e3662868c331232f75a93d4b8a1797bc8ddb5/CHANGELOG.rst#breaking-api-changes). Security: - Update dependency versions for security. License scanning: - Add 22 new and update 71 existing reference licenses - Update licenses to include the SPDX license list 3.12 - Improve license detection accuracy with over 2300 new and improved license detection rules - Undeprecate the regexp license and deprecate the hs-regexp-orig license - Improve license db initial load time with caching for faster scancode start time - Ensure that license short names are no more than 50 characters long - Thank you to: - Dennis Clark @DennisClark - Chin-Yeung Li @chinyeungli - Armijn Hemmel @armijnhemel - Sarita Singh @itssingh - Akanksha Garg @akugarg Copyright scanning: - Detect SPDX-FileCopyrightText as defined by the FSFE Reuse project - Fix bug when using the --filter-clues command line option Thank you to Van Lindberg @VanL - Allow calling copyright detection from text lines to ease integration Thank you to Jelmer Vernooij @jelmer Package scanning: - Add support for installed RPMs detection internally (not wired to scans) Thank you to Chin-Yeung Li @chinyeungli - Improve handling of Debian copyright files with faster and more accurate license detection Thank you to Thomas Druez @tdruez - Add new built-in support for installed_files report. Only available when used as a library. - Improve support for RPM, npm, Debian, build scripts (Bazel) and Go packages Thank you to: - Divyansh Sharma @Divyansh2512 - Jonothan Yang @JonoYang - Steven Esser @majurg - Add new support to collect information from semi-structured Readme files and related metadata files. Thank you to: - Jonothan Yang @JonoYang - Steven Esser @majurg Ouputs: - Add new Debian copyright-formatted output. Thank you to Jelmer Vernooij @jelmer - Fix bug in --include where directories where not skipped correctly Thank you to Pierre Tardy @tardyp Misc. and documentation improvements: - Update the way tests assertions are made Thank you to Aditya Viki @adityaviki - Thank you to Aryan Kenchappagol @aryanxk02 The sources of third-party dependencies are available for download here in https://github.com/nexB/thirdparty-packages/ and in https://github.com/nexB/scancode-plugins. 2021-04-01T15:21:49+00:00 mcsema v3.0.26 mcsema v3.0.26 2021-04-01T19:49:56+00:00 2021-04-01T19:49:56+00:00 tesseract 5.0.0-alpha-20210401 tesseract 5.0.0-alpha-20210401 2021-04-01T20:23:57+00:00 This is a new pre-release of Tesseract 5.0.0. * Replaced all remaining `STRING` by `std::string` * Replaced lots of `GenericVector` by `std::vector` * Replaced all `malloc` / `free` by C++ code * Modernized and formatted code See also list of [all changes](https://github.com/tesseract-ocr/tesseract/compare/5.0.0-alpha-20201231...5.0.0-alpha-20210401). 2021-04-01T20:23:57+00:00 caddy v2.4.0-beta.2 caddy v2.4.0-beta.2 2021-04-02T00:21:55+00:00 The beta channel marches on for 2.4 with beta 2. In addition to the changes/additions/improvements from the previous beta, this release includes: - Caddyfile: global option to configure loggers - Content negotation: precompressed sidecar files (static file server) - Dark mode in static file server's "browse" directory listings - Caddyfile: New `error` directive to return internal error values in HTTP handler chain - Caddyfile: Configure other apps from global options of HTTP Caddyfile - :warning: reverse_proxy: Deprecate `health_path` in favor of `health_uri` for active health checks - reverse_proxy: New placeholders related to latency and duration - Fixed DNS challenge for several providers. ## Changelog a87f757f .gitignore: add IDE files (#4087) 28a41599 CONTRIBUTING: fix spelling (#4070) aac1ccf1 caddy: Add InstanceID() method ec3ac840 caddy: Support SetReadBuffer and SyscallConn for QUIC (fix #3998) 2a127ac3 caddyconfig: add global option for configuring loggers (#4028) d63d5ae1 caddyhttp: improve grammar of comment for AllowH2C (#4072) ec309c6d caddypki: Add SignWithRoot option for ACME server f6bb02b3 caddytls: Remove old asset migration code (close #3894) 911c8a37 cmd: Use formatted logger for config adapter warnings (#4080) 87fbc078 cmd: main: fix minor doc typos (#4082) a8fdc0a9 core: Initialize logging before admin f35a7fa4 encode,staticfiles: Content negotiation, precompressed files (#4045) 0018b9be fileserver: Add a few more debug lines (#4063) a48c6205 fileserver: Browse listing supports dark mode (#4066) 5bf0a55d fileserver: Don't replace in request paths (fix #4027) 427bbe99 go.mod: Latest CertMagic (updated libdns conventions) 1c8ea008 go.mod: Migrate to golang.org/x/term (#4073) a0a7c60c go.mod: Use latest CertMagic f1c36680 headers: Fix Caddyfile parsing for `request_header` with matchers (#4085) 0d7fe360 httpcaddyfile: Add `error` directive for the existing handler (#4034) bafb5629 httpcaddyfile: Configure other apps from global options (#3990) edb362aa httpcaddyfile: Fix catch-all site block sorting f137b822 logging: add replace filter for static value replacement (#4029) 802f80c3 map: Accept regex substitution in outputs (#3991) 6722ae3a reverseproxy: Add duration/latency placeholders (close #4012) (#4013) ce5a0934 reverseproxy: Fix round robin data race (#4038) 51f35ba0 reverseproxy: Fix upstreams with placeholders with no port (#4046) 75f797de reverseproxy: Implement health_uri, deprecate health_path, supports query (#4050) bd357bf0 reverseproxy: Set cookie path to `/` when using cookie lb_policy (#4096) ad8d01cb rewrite: Implement regex path replacements a6bc5815 sigtrap_posix: add missing comma to SIGTERM info (#4078) 2021-04-02T00:21:55+00:00 srum-dump 2.3 srum-dump 2.3 2021-04-02T01:12:03+00:00 This is just a rebuild of the previous version of code with an update to underlying modules. The old version would crash with an error about not being able to read the number of records because of an error in one of the underlying modules. Initial testing of this rebuild shows that it does not seem to suffer form the same problem. 2021-04-02T01:12:03+00:00 tidb v4.0.12 tidb v4.0.12 2021-04-02T02:43:01+00:00 ## Improvements * Refine the output information of the `EXPLAIN` statement for the `batch cop` mode [#23164](https://github.com/pingcap/tidb/pull/23164) * Add the warning information for expressions that cannot be pushed to the storage layer in the output of the `EXPLAIN` statement [#23020](https://github.com/pingcap/tidb/pull/23020) * Migrate a part of the DDL package code from `Execute`/`ExecRestricted` to the safe API (2) [#22935](https://github.com/pingcap/tidb/pull/22935) * Migrate a part of the DDL package code from `Execute`/`ExecRestricted` to the safe API (1) [#22929](https://github.com/pingcap/tidb/pull/22929) * Add `optimization-time` and `wait-TS-time` into the slow log [#22918](https://github.com/pingcap/tidb/pull/22918) * Support querying `partition_id` from the `infoschema.partitions` table [#22489](https://github.com/pingcap/tidb/pull/22489) * Add `last_plan_from_binding` to help the users know whether a SQL statement's execution plan is matched with the hints in the binding [#21430](https://github.com/pingcap/tidb/pull/21430) * Scatter truncated tables without the `pre-split` option [#22872](https://github.com/pingcap/tidb/pull/22872) * Add three format specifiers for the `str_to_date` expression [#22812](https://github.com/pingcap/tidb/pull/22812) * Record the `PREPARE` execution failure as `Failed Query OPM` in the metrics monitor [#22672](https://github.com/pingcap/tidb/pull/22672) * Do not report errors for the `PREPARE` execution if `tidb_snapshot` is set [#22641](https://github.com/pingcap/tidb/pull/22641) ## Bug Fixes * Fix the issue that the `get` variable expression goes wrong when the session variable is hexadecimal literals[#23372](https://github.com/pingcap/tidb/pull/23372) * Fix the issue that wrong collation is used when creating the fast execution plan for the `Enum` or `Set` type [#23292](https://github.com/pingcap/tidb/pull/23292) * Fix the possible wrong result of the `nullif` expression when it is used with `is-null` [#23279](https://github.com/pingcap/tidb/pull/23279) * Fix the issue that the auto-analysis is triggered outside its time range [#23219](https://github.com/pingcap/tidb/pull/23219) * Fix the issue that the `CAST` function might ignore errors for the `point get` plan [#23211](https://github.com/pingcap/tidb/pull/23211) * Fix a bug that prevents SPM from taking effect when `CurrentDB` is empty [#23209](https://github.com/pingcap/tidb/pull/23209) * Fix the issue of possible wrong table filters for the IndexMerge plan [#23165](https://github.com/pingcap/tidb/pull/23165) * Fix the issue of unexpected `NotNullFlag` in the returning types of the `NULL` constant [#23135](https://github.com/pingcap/tidb/pull/23135) * Fix a bug that collation might not be handled by the text type [#23092](https://github.com/pingcap/tidb/pull/23092) * Fix the issue that the range partition might incorrectly handle the `IN` expression [#23074](https://github.com/pingcap/tidb/pull/23074) * Fix the issue that after marking a TiKV store as tombstone, starting new TiKV stores with different StoreIDs with the same IP address and port keeps returning the `StoreNotMatch` error [#23071](https://github.com/pingcap/tidb/pull/23071) * Do not adjust the `INT` type when it is `NULL` and compared with `YEAR` [#22844](https://github.com/pingcap/tidb/pull/22844) * Fix the issue of lost connection when loading data on tables with the `auto_random` column [#22736](https://github.com/pingcap/tidb/pull/22736) * Fix the issue of DDL hangover when the DDL operation meets panic in the cancelling path [#23297](https://github.com/pingcap/tidb/pull/23297) * Fix the wrong key range of index scan when comparing the `YEAR` column with `NULL` [#23104](https://github.com/pingcap/tidb/pull/23104) * Fix the issue that a successfully created view is failed to use [#23083](https://github.com/pingcap/tidb/pull/23083) 2021-04-02T02:43:01+00:00 celery v5.1.0b1 celery v5.1.0b1 2021-04-02T04:55:05+00:00 This document contains change notes for bugfix & new features in 5.1.x series. 5.1.0b1 is an incremental pre-release with lots of bug fixes and some new features/enhancements. Some dependencies were upgraded to newer versions. .. _version-5.1.0b1: 5.1.0b1 ======= :release-date: 2021-04-02 10.55 P.M UTC+6:00 :release-by: Asif Saif Uddin - Add sentinel_kwargs to Redis Sentinel docs. - Depend on the maintained python-consul2 library. (#6544). - Use result_chord_join_timeout instead of hardcoded default value. - Upgrade AzureBlockBlob storage backend to use Azure blob storage library v12 (#6580). - Improved integration tests. - pass_context for handle_preload_options decorator (#6583). - Makes regen less greedy (#6589). - Pytest worker shutdown timeout (#6588). - Exit celery with non zero exit value if failing (#6602). - Raise BackendStoreError when set value is too large for Redis. - Trace task optimizations are now set via Celery app instance. - Make trace_task_ret and fast_trace_task public. - reset_worker_optimizations and create_request_cls has now app as optional parameter. - Small refactor in exception handling of on_failure (#6633). - Fix for issue #5030 "Celery Result backend on Windows OS". - Add store_eager_result setting so eager tasks can store result on the result backend (#6614). - Allow heartbeats to be sent in tests (#6632). - Fixed default visibility timeout note in sqs documentation. - Support Redis Sentinel with SSL. - Simulate more exhaustive delivery info in apply(). - Start chord header tasks as soon as possible (#6576). - Forward shadow option for retried tasks (#6655). --quiet flag now actually makes celery avoid producing logs (#6599). - Update platforms.py "superuser privileges" check (#6600). - Remove unused property `autoregister` from the Task class (#6624). - fnmatch.translate() already translates globs for us. (#6668). - Upgrade some syntax to Python 3.6+. - Add `azureblockblob_base_path` config (#6669). - Fix checking expiration of X.509 certificates (#6678). - Drop the lzma extra. - Fix JSON decoding errors when using MongoDB as backend (#6675). - Allow configuration of RedisBackend's health_check_interval (#6666). - Safeguard against schedule entry without kwargs (#6619). - Docs only - SQS broker - add STS support (#6693) through kombu. - Drop fun_accepts_kwargs backport. - Tasks can now have required kwargs at any order (#6699). - Min py-amqp 5.0.6. - min billiard is now 3.6.4.0. - Minimum kombu now is5.1.0b1. - Numerous docs fixes. - Moved CI to github action. - Updated deployment scripts. - Updated docker. - Initial support of python 3.9 added. 2021-04-02T04:55:05+00:00 Lookyloo v1.5.0 Lookyloo v1.5.0 2021-04-02T14:02:45+00:00 Once again, many of the changes in this release wouldn't have been possible without the support of [Internews](https://globaltech.internews.org/) and the [BASICS Project](https://globaltech.internews.org/our-resources/basics). And we would also like to thank [Credit Agricole](https://www.credit-agricole.com/en/) and @FafnerKeyZee for the continuous bug reports! The main new feature of this release is the possibility to capture URLs present in a capture you already made, all that while **keeping the context** your browser was in (cookies, user-agent, referer) in the **subsequent captures**. It is especially useful when the page you're landing on expects the user to click on a link in order to load the content, the website checks the referer and/or cookies, and bounce you if you're not presenting the right session. This feature will be further extended in the upcoming releases to allow other types of requests (`POST`), and let the user choose the link(s) to captures from the screenshot of the page itself. This release also adds a new **background indexer** so the captures queued with the API are automatically cached even if they are never opened in the browser. And there are the usual bunch of bugfixes, improvements and dependencies upgrades. And we also require python 3.8+. You can also **search** for hostnames, URLs, hashes, and cookies names from the `/search` entry point. This entry point is *not* listed yet in the documentation, but it will be added soon. 2021-04-02T14:02:45+00:00 dexcalibur v0.7.9 dexcalibur v0.7.9 2021-04-02T20:16:07+00:00 Fixed issues : - #49 , #50 : Make smali parser more resilient with invalid smali Details: - Goto label inserted into array statement are ignored - Addditionnal characters after string delimiter are ignored 2021-04-02T20:16:07+00:00 networkx networkx-2.5.1 networkx networkx-2.5.1 2021-04-03T22:13:05+00:00 2021-04-03T22:13:05+00:00 artifacts 20210404 artifacts 20210404 2021-04-04T09:18:17+00:00 Release of version 20210404 2021-04-04T09:18:17+00:00 seaweedfs 2.37 seaweedfs 2.37 2021-04-05T01:47:10+00:00 * Filer * Add customizable upsertQuery for sql file stores. #1946 #1948 * Add concurrent upload size limit to avoid OOM * Ensure creating tables #1957 * Java client 1.6.4 * ByteBuffer.clear supports higher JDK versions * Volume * Add concurrent upload size limit to avoid OOM 2021-04-05T01:47:10+00:00 PrivateBin 1.3.5 PrivateBin 1.3.5 2021-04-05T15:45:40+00:00 This bug fix releases addresses a number of smaller issues and regressions, adds four new translations and includes updated libraries. Links in pastes now by default open in a new browser tab or window. The project information text and link is now a configuration option. ## Benefits of switching to the new release We recommend to upgrade 1.3.x instances to address these issues. ## Update procedure As usual, you can [download the archive](https://github.com/PrivateBin/PrivateBin/releases/latest) for a manual upgrade and can find more details in the [installation instructions](https://github.com/PrivateBin/PrivateBin/blob/master/INSTALL.md#installation). We also offer a [Docker container](https://hub.docker.com/r/privatebin/nginx-fpm-alpine/) that includes the recommended secure setup with the non-essential files and data outside of the web servers document root. If you have enabled the `fileupload` setting and use a custom `cspheader`, please consider adding `allow-downloads` to the `sandbox` property. This lets users of the Google Chrome browser, version 83 or higher, download attachments - inline display of images, media or PDFs files was not affected by this change in Chrome's sandbox behaviour. ## Changes since version 1.3.4 * ADDED: Translation for Hebrew, Lithuanian, Indonesian and Catalan * ADDED: Make the project info configurable (#681) * CHANGED: Upgrading libraries to: DOMpurify 2.2.7, kjua 0.9.0 & random_compat 2.0.18 * CHANGED: Open all links in new window (#630) * FIXED: PDF display in Firefox (#630) * FIXED: Allow pasting into password input dialog (#630) * FIXED: Display of expiration date in email (#630) * FIXED: Allow display of durations in weeks (#630) * FIXED: Avoid exposing burn-after-reading messages from cache (#630) * FIXED: Only display the dropzone when it should (#630) * FIXED: Detect delete token properly (#630) * FIXED: Sanitize output from `Helper.urls2links()` (#630) * FIXED: Avoid recreation of existing pasteurl element when calling URL shortener (#630) * FIXED: Downloads in Chrome >= 83 (#634) * FIXED: Display of empty files (#663) * FIXED: Improve OpenGraph attributes (#651) * FIXED: Reset to configured burn-after-reading, discussion and expiration settings (#682) * FIXED: Italic segment of project information (#756) More details about the plans for future releases and on how you can help the project achieve them, can be found in the [PrivateBin version 1.3.5 release announcements](https://privatebin.info/news/v1.3.5-release.html). 2021-04-05T15:45:40+00:00 clair v4.1.0-alpha.1 clair v4.1.0-alpha.1 2021-04-05T15:53:21+00:00 <a name="unreleased"></a> ## [Unreleased] <a name="v4.1.0-alpha.1"></a> ## [v4.1.0-alpha.1] - 2021-04-05 ### All - [a5bfaeb](https://github.com/quay/clair/commit/a5bfaeb33cc43350234345aba0059a02098f0d67): switch to using codec package ### Chore - [493beb1](https://github.com/quay/clair/commit/493beb13d3a9d0739bcffa74217f7e2107f8438d): v4.1.0-alpha.1 changelog bump - [4734435](https://github.com/quay/clair/commit/473443575e0160cdc83574dcd48982d9922ddf4e): v4.1.0-alpha.1 changelog bump - [6e8a838](https://github.com/quay/clair/commit/6e8a838305c6bdc6a71e3b3a9ee5735660ebbd22): bump cc to v0.4.0 - [5a6f1c3](https://github.com/quay/clair/commit/5a6f1c3b24f9c178838e905a1435078f9706a7b9): update claircore version for database fix - [ea0378d](https://github.com/quay/clair/commit/ea0378d4d67376ebb924b7fb78d4d4f22ad9e1de): bump cc v0.3.0 - [6e195c9](https://github.com/quay/clair/commit/6e195c99a14139360c8d09f90c94024eb7d27b67): fix yaml file indentation issue ### Cicd - [b1145e3](https://github.com/quay/clair/commit/b1145e3a1c5e8faf3d1a64a403de940386b73102): sort changelog by semver - [7dc55fa](https://github.com/quay/clair/commit/7dc55fa9bb0b968ab580c7d6d0ea4ffa053eaba0): bump in go.16, bump out go1.14 - [d5e57af](https://github.com/quay/clair/commit/d5e57afb594d58cf817a962d9e282c820ab6577e): enable CI on stable branches - [f7737e5](https://github.com/quay/clair/commit/f7737e58cfca3640d4a901a658317becb47ba2af): fix openshift ci/cd script - [30c0311](https://github.com/quay/clair/commit/30c0311a8b1584a40f5b956b3b3d9e9ab7eee18a): update golang container for go-mod in app-sre - [cb656df](https://github.com/quay/clair/commit/cb656dfbd69ff1ce11976c7de672b50277091ab8): add notifier to app interface - [9254ab6](https://github.com/quay/clair/commit/9254ab66ea7f1b9711242026045da35b7ffa2782): use quay.io image in CI and Dockerfile ### Clair - [ecd8999](https://github.com/quay/clair/commit/ecd8999cbfd6b9140f0aa8aebc11a67cbefcb4d2): fix initialization error logging - [dc2f893](https://github.com/quay/clair/commit/dc2f8936a564fbc234e1b8f00a3eb4778452f2ec): reorganize initialization - [391c2f7](https://github.com/quay/clair/commit/391c2f766bcbf9c2392c12dca2bb9f225f1ef424): add Shutdown struct ### Claircore - [f183421](https://github.com/quay/clair/commit/f1834212272b07f02228b04a67e9339001dc51f8): bump to v0.2.0 ### Clairctl - [5740a1b](https://github.com/quay/clair/commit/5740a1b0427c81ae5f447add372db43a1ec73dbf): Add subpath to clairctl ### Client - [bd50a95](https://github.com/quay/clair/commit/bd50a9570d996578e0209286a66ec3d7f41d6aaf): remove request body buffering - [ce11fd7](https://github.com/quay/clair/commit/ce11fd7077c2fb10715b37a8248b42583d930462): fix panic on request failure - [#1186](https://github.com/quay/clair/issues/1186)### Codec - [1fb6dcf](https://github.com/quay/clair/commit/1fb6dcfd32143520aa348b184e865be7a6081134): add package for codec pooling ### Config - [e9390fa](https://github.com/quay/clair/commit/e9390fadc24e53e455360f709e79674f752c4a29): add matchers settings - [eb519e0](https://github.com/quay/clair/commit/eb519e0752d3cf7f5f8daeefd4ad9bd29cbfa8c2): allow gc to be disabled - [f2d7313](https://github.com/quay/clair/commit/f2d731341722e3d59c9351c10b7e8eedbe74f276): rework into specific validators ### Docs - [0f230f9](https://github.com/quay/clair/commit/0f230f99f22150a00b36654ee8a5a7674e5507f7): add support matrix - [102ae88](https://github.com/quay/clair/commit/102ae88dd84c1f769b8c037226d92b301d887aab): update cli reference - [9d0a2b2](https://github.com/quay/clair/commit/9d0a2b20a6808f0e86cbd4f2a6046a6c7abdc2ea): fix psk related config references - [44303dc](https://github.com/quay/clair/commit/44303dccfd26935fd66ff041e22602c709c4a428): install clairctl correctly - [a3bb1b6](https://github.com/quay/clair/commit/a3bb1b6d8caebf228ac39b8793d5326bea0d1b55): use correct clairctl subcommands - Closes [#1122](https://github.com/quay/clair/issues/1122)### Documentation - [2e65925](https://github.com/quay/clair/commit/2e6592500fbe9c3197782133965de6503b07b6ab): modified testing.md for clarity - [#1180](https://github.com/quay/clair/issues/1180)### Httptransport - [21dc720](https://github.com/quay/clair/commit/21dc720a7f1e63e731eadbf72cf192913bf88c39): add mime type to indexer and matcher handler - [8616cc6](https://github.com/quay/clair/commit/8616cc68b030fc417c693b3d2dc7208015ce9f4e): return Accepted when not ready - [1ac26da](https://github.com/quay/clair/commit/1ac26daf5501876495ec09f4e67b50eaca4bd1a5): fix panic in metrics registration - [7305b3d](https://github.com/quay/clair/commit/7305b3d735786e340833e045e2cd5888c8af866b): use correct handler for state endpoint - [df5e7f9](https://github.com/quay/clair/commit/df5e7f9658b1fed55d067013656115b062127c23): check for err before deferring resp.Body.Close() ### Initialize - [8a2df09](https://github.com/quay/clair/commit/8a2df099fe2e69a572e8d81b352f688f82de341a): remove New function - [2d27ae5](https://github.com/quay/clair/commit/2d27ae5cd3fe55737c2fa02b46616ec09ade47c5): add standalone initialization functions ### Instrospection - [b78f954](https://github.com/quay/clair/commit/b78f954dbf3210f7deb87dc371b1d35cba216d78): bump to opentelemetry 0.16.0 ### Introspection - [1ece08f](https://github.com/quay/clair/commit/1ece08f49434828c8c672f08ec45844b99187983): database metrics for notifier - [84ba35f](https://github.com/quay/clair/commit/84ba35f29ee81849cb2f424b3624895f9bd05a79): implement prometheus http ### Local-Dev - [1c85589](https://github.com/quay/clair/commit/1c85589abdef98b5af8d4f6e2cd9eb5db6a723a0): remove unintented change in config.yaml ### Logging - [9f3d167](https://github.com/quay/clair/commit/9f3d167d5d85d345c7d0ee666be075a545a553f4): move to zlog throughout ### Matcher - [858c540](https://github.com/quay/clair/commit/858c540b2ef9b8d7f71d16bbe3ba797f73f654ab): add Initialized method ### Notifier - [e7bf3b1](https://github.com/quay/clair/commit/e7bf3b1730e04ad10ec4baef1643556bf5626090): construct notification objects directly - [9962202](https://github.com/quay/clair/commit/99622021c594149a0b0d183b6349e2ee7139e5d2): do AffectedManifests calls in chunks ### Severity_mapping - [8e39fa4](https://github.com/quay/clair/commit/8e39fa40eebca7b50ab29f0001686fa7c5c49e1e): remove defcon1 severity ### Updaters - [8105b03](https://github.com/quay/clair/commit/8105b033fb53f0907373f6af76af954fe95a856d): plumb update retention in [Unreleased]: https://github.com/quay/clair/compare/v4.1.0-alpha.1...HEAD [v4.1.0-alpha.1]: https://github.com/quay/clair/compare/v4.0.4...v4.1.0-alpha.1 2021-04-05T15:53:21+00:00 asciimatics 1.13.0 asciimatics 1.13.0 2021-04-05T16:37:38+00:00 * Added ability to change a Button's text through a .text attribute. * Added ability to accept a name attribute in the Button and Label constructors. * Added ability to detect job pause/resume and force full screen refresh. * Added ability to request terminal default colours using Screen.COLOUR_DEFAULT. * Converted widgets to a sub-package. * Fixed issue with labels in a layout column preventing buttons from being pressed. * Fixed issue with visual overrun on Listboxes when there is a label offset. * Fixed issue with TextBox hitting IndexError in double buffers due to lack of clipping. * Fixed issue with Text/TextBox start columns on reset. * Added troubleshooting on terminal colour handling. 2021-04-05T16:37:38+00:00 gopass v1.12.6-rc0 gopass v1.12.6-rc0 2021-04-05T19:26:09+00:00 ## 1.12.5 / 2021-03-27 * [BUGFIX] Allow subkeys (#1843, #1841, #1842) * [BUGFIX] Avoid logging credentials (#1886, #1883) * [BUGFIX] Fix SSH Command override on termux (#1881) * [CLEANUP] Moving pkg/pinentry to gopasspw/pinentry (#1876) * [ENHANCEMENT] Add -f flag to create (#1867, #1811) * [ENHANCEMENT] Add gopass ln (#1828) * [ENHANCEMENT] Add proper diff numbers on sync (#1882) * [ENHANCEMENT] Update password rules (#1861) 2021-04-05T19:26:09+00:00 cytoscape.js v3.17.4 cytoscape.js v3.17.4 2021-04-05T21:46:53+00:00 2021-04-05T21:46:53+00:00 cytoscape.js v3.18.2 cytoscape.js v3.18.2 2021-04-05T22:01:39+00:00 2021-04-05T22:01:39+00:00 chipsec 1.6.1 chipsec 1.6.1 2021-04-05T22:50:30+00:00 __New or Updated Modules/UtilCmds:__ * None __New or Updated Configurations:__ * `8086/pch_5xxh` - Fix missing type attribute __Removed Modules:__ * None __Additional Changes:__ * `chipsec.chipset` - Update exception logic from cs exception changes __Additional Notes:__ * __For Windows users, please rebuild your windows driver as updates have been made.__ * __For Windows, Linux and MacOS: Python2 support has been deprecated as of June 2020. Please use Python3.__ * Any modules under the `modules.tools` directory have not yet been fully validated to work with Python3. __New Module Details:__ Module Name | Supported Platforms | Flags :---: | :---: | :---: 2021-04-05T22:50:30+00:00 syncthing v1.15.0 syncthing v1.15.0 2021-04-06T06:17:00+00:00 This release fixes a vulnerability where Syncthing and the relay server can crash due to malformed relay protocol messages (CVE-2021-21404); see https://github.com/syncthing/syncthing/security/advisories/GHSA-x462-89pf-6r5h. This release updates the CLI to use subcommands and adds the subcommands `cli` (previously standalone `stcli` utility) and `decrypt` (for offline verifying and decrypting encrypted folders). With this release we invite everyone to test the "untrusted (encrypted) devices" feature. You should not use it yet on important production data. Thus UI controls are hidden behind a feature flag. For more information, visit: https://forum.syncthing.net/t/testing-untrusted-encrypted-devices/16470 Bugfixes: - #7198: Removing a folder and re-adding it again (untrusted) makes it not sync until restart - #7385: infinite filesystem recursion due to missing file id - #7419: panic "Snapshot(): database is closed" on start after wiping database - #7429: "Ignore Patterns" tab disappears after visiting "Folder Defaults" - #7434: Non-DNS-like computer name fails creation of HTTPS certificate - #7451: strelaysrv v1.14.0 failing to load - #7466: Receive-encrypted fails after a while: file with empty block list - #7469: .stignore should be ignored in Receive Encrypted folders - #7474: Device tries to push ignored files to a remote device in a 3-device setup - #7481: Versioning help strings are not displayed in the GUI - #7509: Sharing the first folder to an untrusted device on an established connection sends plain-text Enhancements: - #6566: Ship stcli in main package - #7457: Remove sub-second precision from the REST API rest/stats/device. - #7514: Increase default strelaysrv network buffer size Other issues: - #4719: Package stcli as Debian package - #7199: Build script -no-upgrade handling is unintuitive 2021-04-06T06:17:00+00:00 seaweedfs 2.38 seaweedfs 2.38 2021-04-06T06:30:24+00:00 * Mount * Add Read Only mode #1961 * Filer * Fix possible nil chunk when uploading #1966 2021-04-06T06:30:24+00:00 syncthing v1.15.1 syncthing v1.15.1 2021-04-06T08:51:44+00:00 This is a patch release to correct the inadvertently changed `--config` flag. Bugfixes: - #7532: Removing pending folders is flaky - #7549: Flag --config was renamed to --conf in 1.15.0 2021-04-06T08:51:44+00:00 Gaffer gaffer2-1.16.0 Gaffer gaffer2-1.16.0 2021-04-06T10:19:08+00:00 ## :sparkles: New Features - ProxyStore can't initialise if the remote store supports operations not on the classpath of the proxy [#2375](https://github.com/gchq/Gaffer/issues/2375) - Allow users to configure which walks are returned in a GetWalks operation [#1754](https://github.com/gchq/Gaffer/issues/1754) ## :robot: Automation - Remote staging failing consistently [#2397](https://github.com/gchq/Gaffer/issues/2397) ## :heart: Contributors We'd like to thank all the contributors who worked on this release! - [@m29827](https://github.com/m29827) - [@d47853](https://github.com/d47853) 2021-04-06T10:19:08+00:00 ChameleonMini Build-29f3d85294691c8274089436c347a30e4db5d10e ChameleonMini Build-29f3d85294691c8274089436c347a30e4db5d10e 2021-04-06T13:43:47+00:00 Built at commit 29f3d85294691c8274089436c347a30e4db5d10e from fptrs 2021-04-06T13:43:47+00:00 fzf 0.27.0 fzf 0.27.0 2021-04-06T13:57:45+00:00 - More border options for `--preview-window` ```sh fzf --preview 'cat {}' --preview-window border-left fzf --preview 'cat {}' --preview-window border-left --border horizontal fzf --preview 'cat {}' --preview-window top:border-bottom fzf --preview 'cat {}' --preview-window top:border-horizontal ``` - Automatically set `/dev/tty` as STDIN on execute action ```sh # Redirect /dev/tty to suppress "Vim: Warning: Input is not from a terminal" # ls | fzf --bind "enter:execute(vim {} < /dev/tty)" # "< /dev/tty" part is no longer needed ls | fzf --bind "enter:execute(vim {})" ``` - Bug fixes and improvements - Signed and notarized macOS binaries (Huge thanks to [BACKERS.md](https://github.com/junegunn/junegunn/blob/main/BACKERS.md)!) 2021-04-06T13:57:45+00:00 uBlock 1.34.1b3 uBlock 1.34.1b3 2021-04-06T14:16:33+00:00 [Commits to master since this release](https://github.com/gorhill/uBlock/compare/1.34.1b3...master) To install the developer build: - **Firefox**: Click [uBlock0_1.34.1b3.firefox.signed.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1b3/uBlock0_1.34.1b3.firefox.signed.xpi). - [uBO works best on Firefox](https://github.com/gorhill/uBlock/wiki/uBlock-Origin-works-best-on-Firefox). - **Chromium**: Install from the Chrome Web Store (CWS): <https://chrome.google.com/webstore/detail/ublock-origin-dev-build/cgbcahbpdhpcegmbfconppldiemgcoii>. - **Thunderbird**: Download [uBlock0_1.34.1b3.thunderbird.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1b3/uBlock0_1.34.1b3.thunderbird.xpi), then drag-n-drop it into Thunderbird's _Add-ons Manager_ pane (Thunderbird 65+ required) --- ## Closed as fixed - [json-prune doesn't remove all properties by full wildcard](https://github.com/uBlockOrigin/uBlock-issues/issues/1545) ## Notable commits without en entry in the issue tracker - [Fix regex used to extract attribute name in element picker](https://github.com/gorhill/uBlock/commit/9a94ba0a226f8d9589806e07409ee6de4925a721) - [Add support for removal of response headers](https://github.com/gorhill/uBlock/commit/f876b68171ff307f27601225607a6801f400437d) 2021-04-06T14:16:33+00:00 netbox v2.11-beta1 netbox v2.11-beta1 2021-04-06T16:04:22+00:00 ## v2.11-beta1 (2021-04-06) **WARNING:** This is a beta release and is not suitable for production use. It is intended for development and evaluation purposes only. No upgrade path to the final v2.11 release will be provided from this beta, and users should assume that all data entered into the application will be lost. **Note:** NetBox v2.11 is the last major release that will support Python 3.6. Beginning with NetBox v2.12, Python 3.7 or later will be required. ### Breaking Changes * All objects now use numeric IDs in their UI view URLs instead of slugs. You may need to update external references to NetBox objects. (Note that this does _not_ affect the REST API.) * The UI now uses numeric IDs when filtering object lists. You may need to update external links to filtered object lists. (Note that the slug- and name-based filters will continue to work, however the filter selection fields within the UI will not be automatically populated.) * The RackGroup model has been renamed to Location (see [#4971](https://github.com/netbox-community/netbox/issues/4971)). Its REST API endpoint has changed from `/api/dcim/rack-groups/` to `/api/dcim/locations/`. * The foreign key field `group` on dcim.Rack has been renamed to `location`. * The foreign key field `site` on ipam.VLANGroup has been replaced with the `scope` generic foreign key (see [#5284](https://github.com/netbox-community/netbox/issues/5284)). * Custom script ObjectVars no longer support the `queryset` parameter: Use `model` instead (see [#5995](https://github.com/netbox-community/netbox/issues/5995)). ### New Features #### Journaling Support ([#151](https://github.com/netbox-community/netbox/issues/151)) NetBox now supports journaling for all primary objects. The journal is a collection of human-generated notes and comments about an object maintained for historical context. It supplements NetBox's change log to provide additional information about why changes have been made or to convey events which occur outside NetBox. Unlike the change log, in which records typically expire after some time, journal entries persist for the life of the associated object. #### Parent Interface Assignments ([#1519](https://github.com/netbox-community/netbox/issues/1519)) Virtual interfaces can now be assigned to a "parent" physical interface by setting the `parent` field on the interface object. This is helpful for associating subinterfaces with their physical counterpart. For example, you might assign virtual interfaces Gi0/0.100 and Gi0/0.200 as children of the physical interface Gi0/0. #### Pre- and Post-Change Snapshots in Webhooks ([#3451](https://github.com/netbox-community/netbox/issues/3451)) In conjunction with the newly improved change logging functionality ([#5913](https://github.com/netbox-community/netbox/issues/5913)), outgoing webhooks now include both pre- and post-change representations of the modified object. These are available in the rendering context as a dictionary named `snapshots` with keys `prechange` and `postchange`. For example, here are the abridged snapshots resulting from renaming a site and changing its status: ```json "snapshots": { "prechange": { "name": "Site 1", "slug": "site-1", "status": "active", ... }, "postchange": { "name": "Site 2", "slug": "site-2", "status": "planned", ... } } ``` Note: The pre-change snapshot for a newly created will always be null, as will the post-change snapshot for a deleted object. #### Mark as Connected Without a Cable ([#3648](https://github.com/netbox-community/netbox/issues/3648)) Cable termination objects (circuit terminations, power feeds, and most device components) can now be marked as "connected" without actually attaching a cable. This helps simplify the process of modeling an infrastructure boundary where we don't necessarily know or care what is connected to an attachment point, but still need to reflect the termination as being occupied. In addition to the new `mark_connected` boolean field, the REST API representation of these objects now also includes a read-only boolean field named `_occupied`. This conveniently returns true if either a cable is attached or `mark_connected` is true. #### Allow Assigning Devices to Locations ([#4971](https://github.com/netbox-community/netbox/issues/4971)) Devices can now be assigned to locations (formerly known as rack groups) within a site without needing to be assigned to a particular rack. This is handy for assigning devices to rooms or floors within a building where racks are not used. The `location` foreign key field has been added to the Device model to support this. #### Dynamic Object Exports ([#4999](https://github.com/netbox-community/netbox/issues/4999)) When exporting a list of objects in NetBox, users now have the option of selecting the "current view". This will render CSV output matching the current configuration of the table being viewed. For example, if you modify the sites list to display only the site name, tenant, and status, the rendered CSV will include only these columns, and they will appear in the order chosen. The legacy static export behavior has been retained to ensure backward compatibility for dependent integrations. However, users are strongly encouraged to adapt custom export templates where needed as this functionality will be removed in v2.12. #### Variable Scope Support for VLAN Groups ([#5284](https://github.com/netbox-community/netbox/issues/5284)) In previous releases, VLAN groups could be assigned only to a site. To afford more flexibility in conveying the true scope of an L2 domain, a VLAN group can now be assigned to a region, site group (new in v2.11), site, location, or rack. VLANs assigned to a group will be available only to devices and virtual machines which exist within its scope. For example, a VLAN within a group assigned to a location will be available only to devices assigned to that location (or one of its child locations), or to a rack within that location. #### New Site Group Model ([#5892](https://github.com/netbox-community/netbox/issues/5892)) This release introduces the new SiteGroup model, which can be used to organize sites similar to the existing Region model. Whereas regions are intended for geographically arranging sites into countries, states, and so on, the new site group model can be used to organize sites by functional role or other arbitrary classification. Using regions and site groups in conjunction provides two dimensions along which sites can be organized, offering greater flexibility to the user. #### Improved Change Logging ([#5913](https://github.com/netbox-community/netbox/issues/5913)) The ObjectChange model (which is used to record the creation, modification, and deletion of NetBox objects) now explicitly records the pre-change and post-change state of each object, rather than only the post-change state. This was done to present a more clear depiction of each change being made, and to prevent the erroneous association of a previous unlogged change with its successor. #### Provider Network Modeling ([#5986](https://github.com/netbox-community/netbox/issues/5986)) A new provider network model has been introduced to represent the boundary of a network that exists outside the scope of NetBox. Each instance of this model must be assigned to a provider, and circuits can now terminate to either provider networks or to sites. The use of this model will likely be extended by future releases to support overlay and virtual circuit modeling. ### Enhancements * [#4833](https://github.com/netbox-community/netbox/issues/4833) - Allow assigning config contexts by device type * [#5344](https://github.com/netbox-community/netbox/issues/5344) - Add support for custom fields in tables * [#5370](https://github.com/netbox-community/netbox/issues/5370) - Extend custom field support to organizational models * [#5375](https://github.com/netbox-community/netbox/issues/5375) - Add `speed` attribute to console port models * [#5401](https://github.com/netbox-community/netbox/issues/5401) - Extend custom field support to device component models * [#5425](https://github.com/netbox-community/netbox/issues/5425) - Create separate tabs for VMs and devices under the cluster view * [#5451](https://github.com/netbox-community/netbox/issues/5451) - Add support for multiple-selection custom fields * [#5608](https://github.com/netbox-community/netbox/issues/5608) - Add REST API endpoint for custom links * [#5610](https://github.com/netbox-community/netbox/issues/5610) - Add REST API endpoint for webhooks * [#5830](https://github.com/netbox-community/netbox/issues/5830) - Add `as_attachment` to ExportTemplate to control download behavior * [#5891](https://github.com/netbox-community/netbox/issues/5891) - Add `display` field to all REST API serializers * [#5894](https://github.com/netbox-community/netbox/issues/5894) - Use primary keys when filtering object lists by related objects in the UI * [#5895](https://github.com/netbox-community/netbox/issues/5895) - Rename RackGroup to Location * [#5901](https://github.com/netbox-community/netbox/issues/5901) - Add `created` and `last_updated` fields to device component models * [#5971](https://github.com/netbox-community/netbox/issues/5971) - Add dedicated views for organizational models * [#5972](https://github.com/netbox-community/netbox/issues/5972) - Enable bulk editing for organizational models * [#5975](https://github.com/netbox-community/netbox/issues/5975) - Allow partial (decimal) vCPU allocations for virtual machines * [#6001](https://github.com/netbox-community/netbox/issues/6001) - Paginate component tables under device views * [#6038](https://github.com/netbox-community/netbox/issues/6038) - Include tagged objects list on tag view ### Other Changes * [#1638](https://github.com/netbox-community/netbox/issues/1638) - Migrate all primary keys to 64-bit integers * [#5873](https://github.com/netbox-community/netbox/issues/5873) - Use numeric IDs in all object URLs * [#5938](https://github.com/netbox-community/netbox/issues/5938) - Deprecated support for Python 3.6 * [#5990](https://github.com/netbox-community/netbox/issues/5990) - Deprecated `display_field` parameter for custom script ObjectVar and MultiObjectVar fields * [#5995](https://github.com/netbox-community/netbox/issues/5995) - Dropped backward compatibility for `queryset` parameter on ObjectVar and MultiObjectVar (use `model` instead) * [#6014](https://github.com/netbox-community/netbox/issues/6014) - Moved the virtual machine interfaces list to a separate view * [#6071](https://github.com/netbox-community/netbox/issues/6071) - Cable traces now traverse circuits ### REST API Changes * All primary keys are now 64-bit integers * All model serializers now include a `display` field to be used for the presentation of an object to a human user * All device components * Added support for custom fields * Added `created` and `last_updated` fields to track object creation and modification * All device component templates * Added `created` and `last_updated` fields to track object creation and modification * All organizational models * Added support for custom fields * All cable termination models (cabled device components, power feeds, and circuit terminations) * Added `mark_connected` boolean field to force connection status * Added `_occupied` read-only boolean field as common attribute for determining whether an object is occupied * Renamed RackGroup to Location * The `/dcim/rack-groups/` endpoint is now `/dcim/locations/` * circuits.CircuitTermination * Added the `provider_network` field * Removed the `connected_endpoint`, `connected_endpoint_type`, and `connected_endpoint_reachable` fields * circuits.ProviderNetwork * Added the `/api/circuits/provider-networks/` endpoint * dcim.Device * Added the `location` field * dcim.Interface * Added the `parent` field * dcim.PowerPanel * Renamed `rack_group` field to `location` * dcim.Rack * Renamed `group` field to `location` * dcim.Site * Added the `group` foreign key field to SiteGroup * dcim.SiteGroup * Added the `/api/dcim/site-groups/` endpoint * extras.ConfigContext * Added the `site_groups` many-to-many field to track the assignment of ConfigContexts to SiteGroups * extras.CustomField * Added new custom field type: `multi-select` * extras.CustomLink * Added the `/api/extras/custom-links/` endpoint * extras.ExportTemplate * Added the `as_attachment` boolean field * extras.ObjectChange * Added the `prechange_data` field * Renamed `object_data` to `postchange_data` * extras.Webhook * Added the `/api/extras/webhooks/` endpoint * ipam.VLANGroup * Added the `scope_type`, `scope_id`, and `scope` fields (`scope` is a generic foreign key) * Dropped the `site` foreign key field * virtualization.VirtualMachine * `vcpus` has been changed from an integer to a decimal value 2021-04-06T16:04:22+00:00 tidb v5.0.0 tidb v5.0.0 2021-04-07T02:40:05+00:00 ## Compatibility changes ### System variables + Add the [`tidb_executor_concurrency`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_executor_concurrency-new-in-v50) system variable to control the concurrency of multiple operators. The previous `tidb_*_concurrency` settings (such as `tidb_projection_concurrency`) still take effect but with a warning when you use them. + Add the [`tidb_skip_ascii_check`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_skip_ascii_check) system variable to specify whether to skip the ASCII validation check when the ASCII character set is written. This default value is `OFF`. + Add the [`tidb_enable_strict_double_type_check`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_strict_double_type_check-new-in-v50) system variable to determine whether the syntax like `double(N)` can be defined in the table schema. This default value is `OFF`. + Change the default value of [`tidb_dml_batch_size`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_dml_batch_size) from `20000` to `0`. This means that batch DML statements are no longer used by default in `LOAD`/`INSERT INTO SELECT ...`. Instead, large transactions are used to comply with the strict ACID semantics. > **Note:** > > The scope of the variable is changed from session to global, and the default value is changed from `20000` to `0`. If the application relies on the original default value, you need to use the `set global` statement to modify the variable to the original value after the upgrade. + Control temporary tables’ syntax compatibility using the [`tidb_enable_noop_functions`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_noop_functions-new-in-v40) system variable. When this variable value is `OFF`, the `CREATE TEMPORARY TABLE` syntax returns an error. + Add the following system variables to directly control the garbage collection-related parameters: - [`tidb_gc_concurrency`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_gc_concurrency-new-in-v50) - [`tidb_gc_enable`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_gc_enable-new-in-v50) - [`tidb_gc_life_time`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_gc_life_time-new-in-v50) - [`tidb_gc_run_interval`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_gc_run_interval-new-in-v50) - [`tidb_gc_scan_lock_mode`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_gc_scan_lock_mode-new-in-v50) + Change the default value of [`enable-joint-consensus`](https://docs.pingcap.com/tidb/stable/pd-configuration-file.md#enable-joint-consensus-new-in-v50) from `false` to `true`, which enables the Joint Consensus feature by default. + Change the value of [`tidb_enable_amend_pessimistic_txn`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_amend_pessimistic_txn-new-in-v407) from `0` or `1` to `ON` or `OFF`. + Change the default value of [`tidb_enable_clustered_index`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_clustered_index-new-in-v50) from `OFF` to `INT_ONLY` with the following new meanings: + `OFF`: clustered index is enabled. Adding or deleting non-clustered indexes is supported. + `ON`: clustered index is disabled. Adding or deleting non-clustered indexes is supported. + `INT_ONLY`: the default value. The behavior is consistent with that before v5.0. You can control whether to enable clustered index for the INT type together with `alter-primary-key = false`. > **Note:** > > The `INT_ONLY` value of `tidb_enable_clustered_index` in 5.0 GA has the same meaning as the `OFF` value in 5.0 RC. After upgrading from a 5.0 RC cluster with the `OFF` setting to 5.0 GA, it will be displayed as `INT_ONLY`. ### Configuration file parameters + Add the [`index-limit`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file.md#index-limit-new-in-v50) configuration item for TiDB to be compatible with MySQL’s limit on the maximum number of indexes. If the index setting exceeds the default value of this configuration item, when the table schema is re-imported into MySQL, an error is reported. This item’s value defaults to `64` and ranges between `[64,64*8]`. + Add the [`enable-enum-length-limit`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file.md#enable-enum-length-limit-new-in-v50) configuration item for TiDB to be compatible and consistent with MySQL’s ENUM/SET length (ENUM length < 255). The default value is `true`. + Replace the `pessimistic-txn.enable` configuration item with the [`tidb_txn_mode`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_txn_mode) environment variable. + Replace the `performance.max-memory` configuration item with [`performance.server-memory-quota`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file.md#server-memory-quota-new-in-v409) + Replace the `tikv-client.copr-cache.enable` configuration item with [`tikv-client.copr-cache.capacity-mb`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file.md#capacity-mb). If the item’s value is `0.0`, this feature is disabled. If the item’s value is greater than `0.0`, this feature is enabled. Its default value is `1000.0`. + Replace the `rocksdb.auto-tuned` configuration item with [`rocksdb.rate-limiter-auto-tuned`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file.md#rate-limiter-auto-tuned-new-in-v50). + Delete the `raftstore.sync-log` configuration item. By default, written data is forcibly spilled to the disk. Before v5.0, you can explicitly disable `raftstore.sync-log`. Since v5.0, the configuration value is forcibly set to `true`. + Change the default value of the `gc.enable-compaction-filter` configuration item from `false` to `true`. + Change the default value of the `enable-cross-table-merge` configuration item from `false` to `true`. + Change the default value of the [`rate-limiter-auto-tuned`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file.md#rate-limiter-auto-tuned-new-in-v50) configuration item from `false` to `true`. ## New features ### SQL #### List partitioning (**Experimental**) [User document](https://docs.pingcap.com/tidb/stable/partitioned-table.md#list-partitioning) With the list partitioning feature, you can effectively query and maintain tables with a large amount of data. With this feature enabled, partitions and how data is distributed among partitions are defined according to the `PARTITION BY LIST(expr) PARTITION part_name VALUES IN (...)` expression. The partitioned tables’ data set supports at most 1024 distinct integer values. You can define the values using the `PARTITION ... VALUES IN (...)` clause. To enable list partitioning, set the session variable [`tidb_enable_list_partition`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_list_partition-new-in-v50) to `ON`. #### List COLUMNS partitioning (**Experimental**) [User document](https://docs.pingcap.com/tidb/stable/partitioned-table.md#list-columns-partitioning) List COLUMNS partitioning is a variant of list partitioning. You can use multiple columns as partition keys. Besides the integer data type, you can also use the columns in the string, `DATE`, and `DATETIME` data types as partition columns. To enable List COLUMNS partitioning, set the session variable [`tidb_enable_list_partition`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_list_partition-new-in-v50) to `ON`. #### Invisible indexes [User document](https://docs.pingcap.com/tidb/stable/sql-statements/sql-statement-alter-index.md), [#9246](https://github.com/pingcap/tidb/issues/9246) When you tune performance or select optimal indexes, you can set an index to be `Visible` or `Invisible` by using SQL statements. This setting can avoid performing resource-consuming operations, such as `DROP INDEX` and `ADD INDEX`. To modify the visibility of an index, use the `ALTER INDEX` statement. After the modification, the optimizer decides whether to add this index to the index list based on the index visibility. #### `EXCEPT` and `INTERSECT` operators [User document](https://docs.pingcap.com/tidb/stable/functions-and-operators/set-operators.md), [#18031](https://github.com/pingcap/tidb/issues/18031) The `INTERSECT` operator is a set operator, which returns the intersection of the result sets of two or more queries. To some extent, it is an alternative to the `Inner Join` operator. The `EXCEPT` operator is a set operator, which combines the result sets of two queries and returns elements that are in the first query result but not in the second. ### Transaction [User document](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_amend_pessimistic_txn-new-in-v407), [#18005](https://github.com/pingcap/tidb/issues/18005) In the pessimistic transaction mode, if the tables involved in a transaction contain concurrent DDL operations or `SCHEMA VERSION` changes, the system automatically updates the transaction's `SCHEMA VERSION` to the latest to ensure the successful transaction commit, and to avoid that the client receives the `Information schema is changed` error when the transaction is interrupted by DDL operations or `SCHEMA VERSION` changes. This feature is disabled by default. To enable the feature, modify the value of [`tidb_enable_amend_pessimistic_txn`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_amend_pessimistic_txn-new-in-v407) system variable. This feature is introduced in v4.0.7 and has the following issues fixed in v5.0: + The compatibility issue that occurs when TiDB Binlog executes `Add Column` operations + The data inconsistency issue that occurs when using the feature together with the unique index + The data inconsistency issue that occurs when using the feature together with the added index Currently, this feature still has the following incompatibility issues: + Transaction’s semantics might change when there are concurrent transactions + Known compatibility issue that occurs when using the feature together with TiDB Binlog + Incompatibility with `Change Column` ### Character set and collation - Support the `utf8mb4_unicode_ci` and `utf8_unicode_ci` collations. [User document](https://docs.pingcap.com/tidb/stable/character-set-and-collation.md#new-framework-for-collations), [#17596](https://github.com/pingcap/tidb/issues/17596) ### Security [User document](https://docs.pingcap.com/tidb/stable/log-redaction.md), [#18566](https://github.com/pingcap/tidb/issues/18566) To meet security compliance requirements (such as *General Data Protection Regulation*, or GDPR), the system supports desensitizing information (such as ID and credit card number) in the output error messages and logs, which can avoid leaking sensitive information. TiDB supports desensitizing the output log information. To enable this feature, use the following switches: + The global variable [`tidb_redact_log`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_redact_log). Its default value is `0`, which means that desensitization is disabled. To enable desensitization for tidb-server logs, set the variable value to `1`. + The configuration item `security.redact-info-log`. Its default value is `false`, which means that desensitization is disabled. To enable desensitization for tikv-server logs, set the variable value to `true`. + The configuration item `security.redact-info-log`. Its default value is `false`, which means that desensitization is disabled. To enable desensitization for pd-server logs, set the variable value to `true`. + The configuration item `security.redact_info_log` for tiflash-server and `security.redact-info-log` for tiflash-learner. Their default values are both `false`, which means that desensitization is disabled. To enable desensitization for tiflash-server and tiflash-learner logs, set the values of both variables to `true`. This feature is introduced in v5.0. To use the feature, enable the system variable and all configuration items above. ## Performance optimization ### MPP architecture [User document](https://docs.pingcap.com/tidb/stable/tiflash/use-tiflash.md) TiDB introduces the MPP architecture through TiFlash nodes. This architecture allows multiple TiFlash nodes to share the execution workload of large join queries. When the MPP mode is on, TiDB determines whether to send a query to the MPP engine for computation based on the calculation cost. In the MPP mode, TiDB distributes the computation of table joins to each running TiFlash node by redistributing the join key during data calculation (`Exchange` operation), and thus accelerates the calculation. Furthermore, with the aggregation computing feature that TiFlash has already supported, TiDB can pushdown the computation of a query to the TiFlash MPP cluster. Then the distributed environment can help accelerate the entire execution process and dramatically increase the speed of analytic queries. In the TPC-H 100 benchmark test, TiFlash MPP delivers significant processing speed over analytic engines of traditional analytic databases and SQL on Hadoop. With this architecture, you can perform large-scale analytic queries directly on the latest transaction data, with a higher performance than traditional offline analytic solutions. According to the benchmark, with the same cluster resource, TiDB 5.0 MPP shows 2 to 3 times of speedup over Greenplum 6.15.0 and Apache Spark 3.1.1, and some queries have 8 times better performance. Currently, the main features that the MPP mode does not support are as follows (For details, refer to [Use TiFlash](https://docs.pingcap.com/tidb/stable/tiflash/use-tiflash.md)): + Table partitioning + Window Function + Collation + Some built-in functions + Reading data from TiKV + OOM spill This conversation was marked as resolved by yikeke Show conversation + Union + Full Outer Join ### Clustered index [User document](https://docs.pingcap.com/tidb/stable/clustered-indexes.md), [#4841](https://github.com/pingcap/tidb/issues/4841) When you are designing table structures or analyzing database behaviors, it is recommended to use the clustered index feature if you find that some columns with primary keys are often grouped and sorted, queries on these columns often return a certain range of data or a small amount of data with different values, and the corresponding data does not cause read or write hotspot issues. Clustered indexes, also known as _index-organized tables_ in some database management systems, is a storage structure associated with the data of a table. When creating a clustered index, you can specify one or more columns from the table as the keys for the index. TiDB stores these keys in a specific structure, which allows TiDB to quickly and efficiently find the rows associated with the keys, thus improves the performance of querying and writing data. When the clustered index feature is enabled, the TiDB performance improves significantly (for example in the Sysbench test, the read and write performance of TiDB, with clustered index enabled, improves by 58.1%) in the following cases: + When data is inserted, the clustered index reduces one write of the index data from the network. + When a query with an equivalent condition only involves the primary key, the clustered index reduces one read of index data from the network. + When a query with a range condition only involves the primary key, the clustered index reduces multiple reads of index data from the network. + When a query with an equivalent or range condition involves the primary key prefix, the clustered index reduces multiple reads of index data from the network. Each table can either use a clustered or non-clustered index to sort and store data. The differences of these two storage structures are as follows: + When creating a clustered index, you can specify one or more columns in the table as the key value of the index. A clustered index sorts and stores the data of a table according to the key value. Each table can have only one clustered index. If a table has a clustered index, it is called a clustered index table. Otherwise, it is called a non-clustered index table. + When you create a non-clustered index, the data in the table is stored in an unordered structure. You do not need to explicitly specify the key value of the non-clustered index, because TiDB automatically assigns a unique ROWID to each row of data. During a query, the ROWID is used to locate the corresponding row. Because there are at least two network I/O operations when you query or insert data, the performance is degraded compared with clustered indexes. When table data is modified, the database system automatically maintains clustered indexes and non-clustered indexes for you. All primary keys are created as non-clustered indexes by default. You can create a primary key as a clustered index or non-clustered index in either of the following two ways: It is recommended that users use the CLUSTERED and NON-CLUSTERED usage, deliberately leaving out INT_ONLY + Specify the keyword `CLUSTERED | NONCLUSTERED` in the statement when creating a table, then the system creates the table in the specified way. The syntax is as follows: ```sql CREATE TABLE `t` (`a` VARCHAR(255), `b` INT, PRIMARY KEY (`a`, `b`) CLUSTERED); ``` Or ```sql CREATE TABLE `t` (`a` VARCHAR(255) PRIMARY KEY CLUSTERED, `b` INT); ``` You can execute the statement `SHOW INDEX FROM tbl-name` to query whether a table has a clustered index. + Configure the system variable `tidb_enable_clustered_index` to control the clustered index feature. Supported values are `ON`, `OFF`, and `INT_ONLY`. + `ON`: Indicates that the clustered index feature is enabled for all types of primary keys. Adding and dropping non-clustered indexes are supported. + `OFF`: Indicates that the clustered index feature is disabled for all types of primary keys. Adding and dropping non-clustered indexes are supported. + `INT_ONLY`: The default value. If the variable is set to `INT_ONLY` and `alter-primary-key` is set to `false`, the primary keys which consist of single integer columns are created as clustered indexes by default. The behavior is consistent with that of TiDB v5.0 and earlier versions. If a `CREATE TABLE` statement contains the keyword `CLUSTERED | NONCLUSTERED`, the statement overrides the configuration of the system variable and the configuration item. You are recommended to use the clustered index feature by specifying the keyword `CLUSTERED | NONCLUSTERED` in statements. In this way, it is more flexible for TiDB to use all data types of clustered and non-clustered indexes in the system at the same time as required. It is not recommended to use `tidb_enable_clustered_index = INT_ONLY`, because `INT_ONLY` is temporarily used to make this feature compatible and will be deprecated in the future. Limitations for the clustered index are as follows: + Mutual conversion between clustered indexes and non-clustered indexes is not supported. + Dropping clustered indexes is not supported. + Adding, dropping, and altering clustered indexes using `ALTER TABLE` statements are not supported. + Reorganizing and re-creating a clustered index is not supported. + Enabling or disabling indexes is not supported, which means the invisible index feature is not effective for clustered indexes. + Creating a `UNIQUE KEY` as a clustered index is not supported. + Using the clustered index feature together with TiDB Binlog is not supported. After TiDB Binlog is enabled, TiDB only supports creating a single integer primary key as a clustered index. TiDB Binlog does not replicate data changes of existing tables with clustered indexes to the downstream. + Using the clustered index feature together with the attributes `SHARD_ROW_ID_BITS` and `PRE_SPLIT_REGIONS` is not supported. + If the cluster is upgraded to a later version then rolls back, you need to downgrade newly-added tables by exporting table data before the rollback and importing the data after the rollback. Other tables are not affected. ### Async Commit [User document](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_enable_async_commit-new-in-v50), [#8316](https://github.com/tikv/tikv/issues/8316) The client of the database will wait for the database system to complete the transaction commit in two phases (2PC) synchronously. The transaction returns the result to the client after the first phase commit is successful, and the system executes the second phase commit operation in the background asynchronously to reduce the transaction commit latency. If the transaction write involves only one Region, the second phase is omitted directly, and the transaction becomes a one-phase commit. After the Async Commit feature is enabled, with the same hardware and configuration, when Sysbench is set to test the Update index with 64 threads, the average latency decreases by 41.7% from 12.04ms to 7.01ms. When Async Commit feature is enabled, to reduce one network interaction latency and improve the performance of data writes, database application developers are recommended to consider reducing the consistency of transactions from linear consistency to [causal consistency](https://docs.pingcap.com/tidb/stable/transaction-overview.md#causal-consistency). The SQL statement to enable causal consistency is `START TRANSACTION WITH CAUSAL CONSISTENCY`. After the causal consistency is enabled, with the same hardware and configuration, when Sysbench is set to test oltp_write_only with 64 threads, the average latency decreased by 5.6% from 11.86ms to 11.19ms. After the consistency of transactions is reduced from the linear consistency to causal consistency, if there is no interdependence between multiple transactions in the application, the transactions do not have a globally consistent order. **The Async Commit feature is enabled by default for newly created v5.0 clusters.** This feature is disabled by default for clusters upgraded from earlier versions to v5.0. You can enable this feature by executing the `set global tidb_enable_async_commit = ON;` and `set global tidb_enable_1pc = ON;` statements. The limitation for the Async Commit feature is as follows: + Direct downgrade is not supported. ### Enable the Coprocessor cache feature by default [User document](https://docs.pingcap.com/tidb/stable/tidb-configuration-file.md#tikv-clientcopr-cache-new-in-v400), [#18028](https://github.com/pingcap/tidb/issues/18028) In 5.0 GA, the Coprocessor cache feature is enabled by default. After this feature is enabled, to reduce the latency of reading data, TiDB caches the calculation results of the operators pushed down to tikv-server in tidb-server. To disable the Coprocessor cache feature, you can modify the `capacity-mb` configuration item of `tikv-client.copr-cache` to `0.0`. ### Improve the execution performance of `delete * from table where id <? Limit ?` statement [#18028](https://github.com/pingcap/tidb/issues/18028) The p99 performance of the `delete * from table where id <? limit ?` statement is improved by 4 times. ### Optimize load base split strategy to solve the performance problem that data cannot be split in some small table hotspot read scenarios [#18005](https://github.com/pingcap/tidb/issues/18005) ## Improve stability ### Optimize the performance jitter issue caused by imperfect scheduling [#18005](https://github.com/pingcap/tidb/issues/18005) #### Enable the GC Compaction Filter feature by default to reduce GC’s consumption of CPU and I/O resources [User document](https://docs.pingcap.com/tidb/stable/garbage-collection-configuration.md#gc-in-compaction-filter), [#18009](https://github.com/pingcap/tidb/issues/18009) When TiDB performs garbage collection (GC) and data compaction, partitions occupy CPU and I/O resources. Overlapping data exists during the execution of these two tasks. To reduce GC’s consumption of CPU and I/O resources, the GC Compaction Filter feature combines these two tasks into one and executes them in the same task. This feature is enabled by default. You can disable it by configuring `gc.enable-compaction-filter = false`. ### Ensure that the execution plans are unchanged as much as possible to avoid performance jitter [User document](https://docs.pingcap.com/tidb/stable/sql-plan-management.md) #### SQL Binding supports the `INSERT`、`REPLACE`、`UPDATE`、`DELETE` statements When tuning performance or maintaining the database, if you find that the system performance is unstable due to unstable execution plans, you can select a manually optimized SQL statement according to your judgement or tested by `EXPLAIN ANALYZE`. You can bind the optimized SQL statement to the SQL statement to be executed in the application code to ensure stable performance. When manually binding SQL statements using the SQL BINDING statement, you need to ensure that the optimized SQL statement has the same syntax as the original SQL statement. You can view the manually or automatically bound execution plan information by running the `SHOW {GLOBAL | SESSION} BINDINGS` command. The output is the same as that of versions earlier than v5.0. #### Automatically capture and bind execution plans When upgrading TiDB, to avoid performance jitter, you can enable the baseline capturing feature to allow the system to automatically capture and bind the latest execution plan and store it in the system table. After TiDB is upgraded, you can export the bound execution plan by running the `SHOW GLOBAL BINDING` command and decide whether to delete these plans. This feature is disbled by default. You can enable it by modifying the server or setting the `tidb_capture_plan_baselines` global system variable to `ON`. When this feature is enabled, the system fetches the SQL statements that appear at least twice from the Statement Summary every `bind-info-lease` (the default value is `3s`), and automatically captures and binds these SQL statements. ### Improve stability of TiFlash queries Add a system variable [`tidb_allow_fallback_to_tikv`](https://docs.pingcap.com/tidb/stable/system-variables.md#tidb_allow_fallback_to_tikv-new-in-v50) to fall back queries to TiKV when TiFlash fails. The default value is `OFF`. ### Optimize the memory management module to reduce system OOM risks Track the memory usage of aggregate functions. This feature is enabled by default. When SQL statements with aggregate functions are executed, if the total memory usage of the current query exceeds the threshold set by [`mem-quota-query`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file.md#mem-quota-query), the system automatically performs operations defined by [`oom-action`](https://docs.pingcap.com/tidb/stable/tidb-configuration-file.md#oom-action). ## Diagnostics [User document](https://docs.pingcap.com/tidb/stable/sql-statements/sql-statement-explain.md#explain) During the troubleshooting of SQL performance issues, detailed diagnostic information is needed to determine the causes of performance issues. Before TiDB 5.0, the information collected by the `EXPLAIN` statements was not detailed enough. The root causes of the issues can only be determined based on log information, monitoring information, or even on guess, which might be inefficient. In TiDB v5.0, the following improvements are made to help you troubleshoot performance issues more efficiently: + Support using the `EXPLAIN ANALYZE` statement to analyze all DML statements to show the actual performance plans and the execution information of each operator. [#18056](https://github.com/pingcap/tidb/issues/18056) + Support using the `EXPLAIN FOR CONNECTION` statement to check the real-time status of all the SQL statements being executed. For example, you can use the statement to check the execution duration of each operator and the number of processed rows. [#18233](https://github.com/pingcap/tidb/issues/18233) + Provide more details about the operator execution in the output of the `EXPLAIN ANALYZE` statement, including the number of RPC requests sent by operators, the duration of resolving lock conflicts, network latency, the scanned volume of deleted data in RocksDB, and the hit rate of RocksDB caches. [#18663](https://github.com/pingcap/tidb/issues/18663) + Support automatically recording the detailed execution information of SQL statements in the slow log. The execution information in the slow log is consistent with the output information of the `EXPLAIN ANALYZE` statement, which includes the time consumed by each operator, the number of processed rows, and the number of sent RPC requests. [#15009](https://github.com/pingcap/tidb/issues/15009) ## Telemetry TiDB adds cluster usage metrics in telemetry, such as the number of data tables, the number of queries, and whether new features are enabled. To learn more about details and how to disable this behavior, refer to [telemetry](https://docs.pingcap.com/tidb/stable/telemetry.md). 2021-04-07T02:40:05+00:00 mattermost-server v5.31.3 mattermost-server v5.31.3 2021-04-07T12:29:03+00:00 Mattermost Platform Release v5.31.3 2021-04-07T12:29:03+00:00 openlibrary deploy-2021-04-08 openlibrary deploy-2021-04-08 2021-04-08T14:44:42+00:00 General: - @Yashs911 Fix missing less facet on search page (#4626) - @Yashs911 Add book subtitle to lists (#4784) - @mekarpeles APIs: Add basic /account/loans.json endpoint (#4808) - @jamesachamp [beta] Add basic CRUD to book notes modal form (#4803) - @Sabreen-Parveen Add april carousel to home page (#4953) - @Yashs911 A11y: Fix Lists Page Images Accessibility (#4753) - @Yashs911 Add work subtitle in topline of bookpage (#4960) - @RayBB remove overflow hidden from edit tools (#4956) - @RayBB Fix list description not wrapping long strings (#4975) - @jamesachamp [beta] Make BookNotes edition-level (#4866) - @jamesachamp [beta] Remove submit button for observations in book notes modal (#4948) - @lephemere Add name field to contact form (#4979) - @cdrini Fix /search/inside.json limit causing error (#5001) Dev: - @Sabreen-Parveen Monthly Reads refactor (#4881) - @cclauss Upgrade to Python 3.9 to 3.9.2 (#4882) - @jonbos Fix svgo config (#4880) - @jonbos Optimize SVG files (#4890) - @navneetsaluja Refactor: Moved some functions out of lib/analytics.html into own JS (#4860) - @navneetsaluja Refactor: Removed dead reference to `.more_search reference` (#4923) - @cclauss Refactor: Use requests in openlibrary/plugins/upstream/utils.py (#4895) - @cclauss Fix Nothing is not JSON serializable again (#4894) - @cdrini Hotfix: don't carry account variable around (#4945) - @cdrini Remove unneded home service from staging (#4947) - @cdrini Use docker-compose profiles + cleanup old volume-mount cludges (#4936) - @cclauss Refactor: Remove uses of six.PY2 and six.PY3 (#4769) - @drakene Fix for absolute path bug causing local env to break (#4943) - @hornc Importbot -- log actual API error responses (#4891) - @cdrini Add OLDEV_TAG environment variable (#4946) - @cdrini Make python CI not use infogami master (#4973) - @cclauss repr(Nothing()) —> "<Nothing>", not empty str (#4972) - @drakene Refactor: remove inline js for admin (#4922) - @dengliming Upgrade Node.js to v14 (#3791) - @lephemere I18n /contact template (#4754) - @cdrini Make "i18n-messages extract" include more file + regen messages.pot (#5000) - @cclauss Add mypy type checking to our testing (#4988) - @Yashs911 Improve structured data for book name (#4884) Deps: - @dependabot[bot] Bump @babel/preset-env from 7.12.11 to 7.13.10 (#4813) - @jonbos Upgrade css-loader to 5.1.3 (#4921) - @dependabot[bot] Bump gunicorn from 20.0.4 to 20.1.0 (#4918) - @dependabot[bot] Bump sinon from 9.2.4 to 10.0.0 (#4963) - @dependabot[bot] Bump pytest from 6.2.2 to 6.2.3 (#4982) - @dependabot[bot] Bump y18n from 4.0.0 to 4.0.1 (#4959) - @dependabot[bot] Bump webpack-cli from 4.5.0 to 4.6.0 (#4961) - @dependabot[bot] Bump eslint-plugin-vue from 7.7.0 to 7.8.0 (#4967) - @dependabot[bot] Bump core-js from 3.9.1 to 3.10.0 (#4969) Stats: - PR Authors: @cdrini (7), @cclauss (6), @Yashs911 (5), @jamesachamp (3), @jonbos (3), @RayBB (2), @Sabreen-Parveen (2), @drakene (2), @lephemere (2), @navneetsaluja (2), @dengliming (1), @hornc (1), @mekarpeles (1) - PR Assignees: @cdrini (14), @jdlrobson (10), @mekarpeles (10), @cclauss (6), @bpmcneilly (2), @hornc (1), @jamesachamp (1) Full diff: https://github.com/internetarchive/openlibrary/compare/deploy-2021-03-24...deploy-2021-04-08 PRs: [is:pr is:merged merged:2021-03-24T19:22:12Z..2021-04-08T14:26:37Z sort:updated-asc](https://github.com/internetarchive/openlibrary/pulls?q=is%3Apr%20is%3Amerged%20merged%3A2021-03-24T19%3A22%3A12Z..2021-04-08T14%3A26%3A37Z%20sort%3Aupdated-asc) 2021-04-08T14:44:42+00:00 traefik v1.7.30 traefik v1.7.30 2021-04-08T15:18:07+00:00 **Bug fixes:** - **[metrics]** Prometheus: fixing prom Handler to use the custom registry ([#8040](https://github.com/traefik/traefik/pull/8040) by [tomMoulard](https://github.com/tomMoulard)) 2021-04-08T15:18:07+00:00 ILSpy v7.0-rc1 ILSpy v7.0-rc1 2021-04-08T17:33:30+00:00 # General * Dark mode! (big thank you to @tom-englert in #2347) * Building ILSpy for .NET 5 is now possible (see [multitargeting.props.template](https://github.com/icsharpcode/ILSpy/blob/master/multitargeting.props.template#L3)) * Added inline display of CustomDebugInformation table entries in metadata * Add metadata tree nodes for blob, guid, string and user string heap # Contributions * Performance improvements in DataGrid filter (thanks to @tom-englert) * Adjust fitness calculation for destructors (by @MikeFH in #2344) * Refactored insertion of search results (by @MikeFH in #2335) * Use correct URL format in Search MSDN command (by @yyjdelete in #2343) * Warning fixes (by @riQQ in #2345) * Fixed ConnectionIdRewritePass (by @wwh1004 in #2327) # Bug fixes * PDBGen: Ignore duplicated ILFunctions (see 5a8b488) * #2314: ILSpy incorrectly resolves a runtime dependency when dll is present in both WindowsDesktop.App and NETCore.App * #1648 and #2133: Use simple assembly names for KnownThings to allow the resolver to use the relative framework version. * #2349: Use proper ExpressionType with DynamicCompoundAssign. * #1512: XmlDocumentationProvider Unable to load some XML files to have special characters * #2342: Do not generate empty names for foreach loop variables. * #2340: Do not traverse assemblies with load errors in AssemblyList.GetAllAssemblies() * #2356: Can't navigate if switch language in Analyze panel. * Improved decompilation of rethrow/throw and finally blocks in async methods. (see #1749, #2339 and #2353) * And many other fixes, for a full list click [here](https://github.com/icsharpcode/ILSpy/compare/v7.0-preview3...v7.0-rc1). 2021-04-08T17:33:30+00:00 dyninst v11.0.0 dyninst v11.0.0 2021-04-08T21:01:12+00:00 ## Removed deprecated functionality - JumpTableIndexPred::FillInOutEdges - BoundFactCalculator::CheckZeroExtend - BoundsFactCalculator::ThunkBound - binaryEdit::deleteBinaryEdit - CFGFactor::destroy_all - BPatch_regExpr - BPatch_addressSpace::findFunctionByAddr - insnCodeGen::generate - AddressSpace::getAddressWidth - BPatch_process::enableDumpPatchedImage - BPatch_snippet::getCost* - InstrucIter, vectorSet, and lineDict classes - Support for bluegene, vxworks, sparc, and AIX - Replace pdvector with std::vector - The entire serialization infrasturcture ## Minimum version requirements - Boost >= 1.67 - gcc >= 6.0 - Standards-compliant c++11 ABI in lib[std]c++ ## Enhancements - DWARF handling - Fully support separate debug files - Disable parallel parsing of DWARF due to a thread safety issue in elfutils - Support debuginfod - GPUs - Initial support for AMD and Intel GPU code parsing - Code slicing on NVIDIA GPUs - Code cleanup - Substantially reduce number of compiler warnings in public headers - BPatch - Callee lookup defaults to "by name" when other modes fail ## Bug fixes - Symtab - Fix non-deterministic inline function lookup for OpenMP outlined code - Handle C++ name demangling correctly and uniformly - Indirect call analysis - Fix detection of indirect calls to PLT stubs subtly broken by v10.2.1 - Patch memory leaks in emitElf See the complete [CHANGELOG](https://github.com/dyninst/dyninst/blob/481abaf20477a20fc533087010dcb671436de38b/CHANGELOG.md). --- ## Manuals - [DyninstAPI 11.0](https://github.com/dyninst/dyninst/blob/481abaf20477a20fc533087010dcb671436de38b/dyninstAPI/doc/dyninstAPI.pdf) - [PatchAPI 11.0](https://github.com/dyninst/dyninst/blob/481abaf20477a20fc533087010dcb671436de38b/patchAPI/doc/patchAPI.pdf) - [ParseAPI 11.0](https://github.com/dyninst/dyninst/blob/481abaf20477a20fc533087010dcb671436de38b/parseAPI/doc/parseAPI.pdf) - [DataflowAPI 11.0](https://github.com/dyninst/dyninst/blob/481abaf20477a20fc533087010dcb671436de38b/dataflowAPI/doc/dataflowAPI.pdf) - [InstructionAPI 11.0](https://github.com/dyninst/dyninst/blob/481abaf20477a20fc533087010dcb671436de38b/instructionAPI/doc/instructionAPI.pdf) - [SymtabAPI 11.0](https://github.com/dyninst/dyninst/blob/481abaf20477a20fc533087010dcb671436de38b/symtabAPI/doc/symtabAPI.pdf) 2021-04-08T21:01:12+00:00 graudit v2.9 graudit v2.9 2021-04-09T02:15:07+00:00 Fix for GRDIR by @micharu123 Improved C fruit rules Improved js fruit rules Started python fruit rules Updated documentation Fixed typo in dotnet rules (@0x10f2c) Reformatted parts of graudit for readability misc/onlyfiles.sh filescanning with only files matching globs 2021-04-09T02:15:07+00:00 clair v4.1.0-alpha.2 clair v4.1.0-alpha.2 2021-04-09T19:29:26+00:00 <a name="unreleased"></a> ## [Unreleased] <a name="v4.1.0-alpha.2"></a> ## [v4.1.0-alpha.2] - 2021-04-09 ### Chore - [e0eea38](https://github.com/quay/clair/commit/e0eea383b9e791b5b041136b88f1b69b3d4841bb): v4.1.0-alpha.2 changelog bump ### Codec - [d5cac13](https://github.com/quay/clair/commit/d5cac1315481a87f596f395e1c2da2bf57eaf18c): use stdlib time.Time encoding - Closes [#1231](https://github.com/quay/clair/issues/1231)### Docs - [60f9684](https://github.com/quay/clair/commit/60f9684accfd7e6b9e1bd585a55874803e1160f5): minor updates - [cbdc9ca](https://github.com/quay/clair/commit/cbdc9caab450489377ab1d6bb19429d54df639cc): update configuration file reference [Unreleased]: https://github.com/quay/clair/compare/v4.1.0-alpha.2...HEAD [v4.1.0-alpha.2]: https://github.com/quay/clair/compare/v4.1.0-alpha.1...v4.1.0-alpha.2 2021-04-09T19:29:26+00:00 bookmark-archiver v0.6.2 bookmark-archiver v0.6.2 2021-04-10T12:24:06+00:00 ### New features - new ArchiveResult log in the admin web UI, with full editing ability of individual extractor outputs + list of outputs under each Snapshot admin entry - ability to save multiple snapshots of the same URL over time using new `Re-snapshot` button - add `init --quick` and `server --quick-init` options to quickly update the db version without doing a full re-init (for users with large archive collections this will make version upgrades a lot faster / less painful) - add new `archivebox setup` command and `archivebox init --setup` flag to aid in automatically installing dependencies and creating a superuser during initial setup - new `SNAPSHOTS_PER_PAGE=40` and `MEDIA_MAX_SIZE=750m` config options - allow hotlinking directly to specific extractor output on the snapshot detail page using URL `#hash` e.g. `/archive/<timestamp>/index.html#git` - add ability to view snapshot matching a given URLs by visiting `/archive/https://example.com/some/url` -> redirects to -> `/archive/<timestamp>/index.html` (also works without scheme `/archive/example.com`) - #660 add ability to tag URLs while adding them via the web UI and via the CLI using `archivebox add --tag=tag1,tag2,tag3 ...` - #659 add back ability to override visual styling with custom HTML and CSS using new config option `CUSTOM_TEMPLATES_DIR` - ability to add and remove multiple tags at once from the snapshot admin using autocompleting dropdown ### Enhancements - lots of performance improvements! (in testing with 100k entries, the main index was brought down from 10-14 second load times to ~110ms once cache warms up) - full text search now works on the public snapshot list - dates and times are now localized to your browser's timezone instead of showing in UTC - integrity and correctness improvements to readability, mercury, warc, and other extractors - video subtitles and description are now added to the full-text search index as well (including youtube's autogenerated transcripts in all languages) - log all errors with full tracebacks to new `data/logs/errors.log` file (so users no longer have to run in --debug mode to see error details) - better `archivebox schedule` logging and changed logfile location to `./logs/schedule.log` - better docker-compose setup experience with sonic config example in `docker-compose.yml` - add Django Debug Toolbar + `djdt_flamegraph` for developers to profile UI performance - add `--overwrite` flag support to `archivebox schedule`, archived urls get added similarly to `add --overwrite` - #644 remove boostrap and jquery remove network requests to CDNs by inlining them instead - #647 allow filtering by ArchiveResult status in the Snapshot admin UI to select only links that have been archived or not archived - #550 kill all orphan child processes after each extractor finishes to prevent dangling chromium/node subprocesses and memory leaks - 3276434 add new `SEARCH_BACKEND_TIMEOUT` config option to tune amount of time search backend can take before it gives up - more diagnostic info added to the Snapshot admin view including most recent status code, content type, detected server, etc - make the order of the table columns, layout, and spacing the same on the public view and private view (also remove DataTable, we're not using it) - better snapshot grid page (faster load times, nicer CSS for tags and cards, more actions supported and metadata shown) - added `Cache-Control` headers to dramatically speed up load times by caching favicons, screenshots, etc. in browsers/upstreams - new project releases page https://releases.archivebox.io and demo url https://demo.archivebox.io ### Bugfixes - #673 fix searching by URL substring in Snapshot admin list - #658 fix Snapshot admin action buttons not working in Safari and some other browsers - #678 fix `AssertionError` error when archivebox would to attempt archive with `CHROME_BINARY=None` when Chrome was not found on host system - #654 fix some issues with sonic attempting to index massive text blobs or binary blobs on some pages and hanging - #674 fix UTF-8 encoding encoding problems with file reading/writing on Windows (supporting a Python pkg on Windows is unreasonably painful ya'll) - #433 fix deleted items sometimes reappearing on next import/update - #473 fix issue preventing use of archivebox python API inside raw REPL (not using archivebox shell) - fix stdin/stdout/stderr handling for some edge cases in Docker/Docker-Compose ![image](https://user-images.githubusercontent.com/511499/114269685-661a9000-99d6-11eb-8bbd-1b370b9eddf0.png) ![image](https://user-images.githubusercontent.com/511499/113627981-3d8b4280-9632-11eb-94fa-06f310e916f5.png) 2021-04-10T12:24:06+00:00 bookmark-archiver v0.6.0 bookmark-archiver v0.6.0 2021-04-10T12:24:06+00:00 ### New features - new ArchiveResult log in the admin web UI, with full editing ability of individual extractor outputs + list of outputs under each Snapshot admin entry - ability to save multiple snapshots of the same URL over time using new `Re-snapshot` button - add `init --quick` and `server --quick-init` options to quickly update the db version without doing a full re-init (for users with large archive collections this will make version upgrades a lot faster / less painful) - add new `archivebox setup` command and `archivebox init --setup` flag to aid in automatically installing dependencies and creating a superuser during initial setup - new `SNAPSHOTS_PER_PAGE=40` and `MEDIA_MAX_SIZE=750m` config options - allow hotlinking directly to specific extractor output on the snapshot detail page using URL `#hash` e.g. `/archive/<timestamp>/index.html#git` - add ability to view snapshot matching a given URLs by visiting `/archive/https://example.com/some/url` -> redirects to -> `/archive/<timestamp>/index.html` (also works without scheme `/archive/example.com`) - #660 add ability to tag URLs while adding them via the web UI and via the CLI using `archivebox add --tag=tag1,tag2,tag3 ...` - #659 add back ability to override visual styling with custom HTML and CSS using new config option `CUSTOM_TEMPLATES_DIR` - ability to add and remove multiple tags at once from the snapshot admin using autocompleting dropdown ### Enhancements - lots of performance improvements! (in testing with 100k entries, the main index was brought down from 10-14 second load times to ~110ms once cache warms up) - full text search now works on the public snapshot list - dates and times are now localized to your browser's timezone instead of showing in UTC - integrity and correctness improvements to readability, mercury, warc, and other extractors - video subtitles and description are now added to the full-text search index as well (including youtube's autogenerated transcripts in all languages) - log all errors with full tracebacks to new `data/logs/errors.log` file (so users no longer have to run in --debug mode to see error details) - better `archivebox schedule` logging and changed logfile location to `./logs/schedule.log` - better docker-compose setup experience with sonic config example in `docker-compose.yml` - add Django Debug Toolbar + `djdt_flamegraph` for developers to profile UI performance - add `--overwrite` flag support to `archivebox schedule`, archived urls get added similarly to `add --overwrite` - #644 remove boostrap and jquery remove network requests to CDNs by inlining them instead - #647 allow filtering by ArchiveResult status in the Snapshot admin UI to select only links that have been archived or not archived - #550 kill all orphan child processes after each extractor finishes to prevent dangling chromium/node subprocesses and memory leaks - 3276434 add new `SEARCH_BACKEND_TIMEOUT` config option to tune amount of time search backend can take before it gives up - more diagnostic info added to the Snapshot admin view including most recent status code, content type, detected server, etc - make the order of the table columns, layout, and spacing the same on the public view and private view (also remove DataTable, we're not using it) - better snapshot grid page (faster load times, nicer CSS for tags and cards, more actions supported and metadata shown) - added `Cache-Control` headers to dramatically speed up load times by caching favicons, screenshots, etc. in browsers/upstreams - new project releases page https://releases.archivebox.io and demo url https://demo.archivebox.io ### Bugfixes - #673 fix searching by URL substring in Snapshot admin list - #658 fix Snapshot admin action buttons not working in Safari and some other browsers - #678 fix `AssertionError` error when archivebox would to attempt archive with `CHROME_BINARY=None` when Chrome was not found on host system - #654 fix some issues with sonic attempting to index massive text blobs or binary blobs on some pages and hanging - #674 fix UTF-8 encoding encoding problems with file reading/writing on Windows (supporting a Python pkg on Windows is unreasonably painful ya'll) - #433 fix deleted items sometimes reappearing on next import/update - #473 fix issue preventing use of archivebox python API inside raw REPL (not using archivebox shell) - fix stdin/stdout/stderr handling for some edge cases in Docker/Docker-Compose ![image](https://user-images.githubusercontent.com/511499/114269685-661a9000-99d6-11eb-8bbd-1b370b9eddf0.png) ![image](https://user-images.githubusercontent.com/511499/113627981-3d8b4280-9632-11eb-94fa-06f310e916f5.png) 2021-04-10T12:24:06+00:00 cutter v2.0.1 cutter v2.0.1 2021-04-10T19:41:01+00:00 This release updates Rizin to 0.2.0 and brings a few additional fixes on top of Cutter 2.0.0. ## Changes * Updated Rizin to [0.2.0](https://github.com/rizinorg/rizin/releases/tag/v0.2.0) * Updated jsdec to 0.2.0 and rz-ghidra to 0.2.0 * Fix path when building appimage, caused by #2574 * Set macOS deployment target to 10.14 Mojave for compatibility. (#2650) * Removed build.sh (#2654) 2021-04-10T19:41:01+00:00 jsPsych v6.3.1 jsPsych v6.3.1 2021-04-10T20:53:00+00:00 This minor release is focused on improvements to the eye tracking features added in version 6.3.0. # New Features - We have [forked WebGazer](https://github.com/jspsych/WebGazer) to make some minor changes to improve compatibility with jsPsych. These changes allow jsPsych to sample eye movements at a faster rate without disrupting jsPsych's display timing. Thanks to @moltaire for identifying this issue (#1700). - WebGazer no longer initializes automatically when the page loads. Instead, the `webgazer-init-camera` plugin will initialize WebGazer and ask for camera access permissions. Added an `auto_initialize` parameter to the WebGazer extension to allow users to initialize on page load if desired. - The sampling interval can be set in the WebGazer extension. By default the extension aims for 30Hz. Faster intervals are probably not possible until browsers provide better interfaces to synchronize processing with video feeds. - Improved the timing measurements reported by WebGazer by ~30 milliseconds. Timing is now based on an estimate of when the video frame was recorded, instead of when WebGazer finished computational work on the video frame. - `jsPsych.pluginAPI.compareKeys` can now handle `null` values: it will return `true` if both key arguments are `null`, and return `false` if one key is `null` and the other is a key code or key character. This means that this function can be used to check for a correct non-response. (#1577) # Changes - Minor formatting and text updates to the `webgazer-init-camera` plugin to center the camera view. - The `webgazer-validate` plugin now records the coordinates of the validation points. - The `webgazer-init-camera` plugin now records the load time for WebGazer to initialize. - Added a `point_size` parameter to calibration and validation plugins, and set the default to a larger value. # Bug Fixes - Fixed an issue where `webgazer_targets` did not record the bounding box of content on the screen if the content was loaded after the trial began. - Fixed problem with the `minimum_valid_rt` parameter in `jsPsych.init` not working when the WebAudio clock is used to measure keyboard RTs (i.e. `rt_method: 'audio'` in `jsPsych.pluginAPI.getKeyboardResponse`). Thanks @kinleyid! (#1675) - Fixed various documentation errors and formatting problems. (#1642, #1641, #1594) 2021-04-10T20:53:00+00:00 asciidoctor v2.0.13 asciidoctor v2.0.13 2021-04-10T22:52:09+00:00 ## Summary This is an important patch release that resolves a significant regression in the include directive introduced in 2.0.11, patches a discovered security vulnerability with the counter attribute directives, fixes numerous bugs in the man page converter, and brings a handful of other fixes and improvements. We strongly recommend upgrading to this version. This release will also be the candidate for making the switch to Asciidoctor 2 upgrade in Antora. In an attempt to fix the detection of a document header inside an include file, the processor was changed in 2.0.11 to remove leading and trailing empty lines in included AsciiDoc content. This turned out to be the wrong decision because it broke documents that relied on these empty lines to separate blocks. It also didn't fully address the problem. That change has been rolled back and the correct fix applied. We also discovered that it was possible to assign a value to an attribute locked by the API using the counter and counter2 attribute directives. This back door has been closed so that the locked state of the attribute is closed. The processor was also updated so non-ASCII characters used as the value of a counter attribute no longer cause the processor to crash. Thanks to a handful of reports filed by the community, we were able to substantially improve the man page output, including adding support for footnotes and page breaks, not mangling formatting macros when transforming section titles to uppercase, and substituting attribute references in the purpose part of the name section. Thanks to @slonopotamus, the Pygments adapter was updated to support the pygments.rb 2 gem in addition to the pygments.rb 1 gem. You're encouraged to upgrade to pygments.rb 2 as it is both maintained and more stable. If you're using Rouge, thanks to @Oblomov, you should now find it easier to extend the built-in adapter to customize the lexer or formatter as the logic to instantiate them have been extracted so they can be overridden. Finally, to silence all log messages when using the API, you can pass the `logger: nil` option. Consult the CHANGELOG to find the full list of changes in this release. During this release cycle, the documentation for the Asciidoctor processor was imported into this repository and is published on the new https://docs.asciidoctor.org site powered by [Antora](https://antora.org). Thanks to @graphitefriction for her incredibly hard work to make that happen. Please note that it will not be possible to use Asciidoctor on Ruby < 2.3 after the 2.0.x release line. Those versions are already unsupported. However, since we had not yet made a change that prevented them from working, we'll wait until 2.1.x to do so. ## Distributions - [RubyGem (asciidoctor)](https://rubygems.org/gems/asciidoctor) Asciidoctor is also packaged for [Fedora](https://apps.fedoraproject.org/packages/rubygem-asciidoctor), [Debian](https://packages.debian.org/sid/asciidoctor), [Ubuntu](https://packages.ubuntu.com/search?keywords=asciidoctor), [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=asciidoctor), [OpenSUSE](https://software.opensuse.org/package/rubygem-asciidoctor), and [Homebrew](http://brewformulas.org/Asciidoctor). You can use the system's package manager to install the package named **asciidoctor**. ## Release meta Released on: 2020-04-10 Released by: @mojavelinux Release beer: Rumpin by Avery Brewing Company Logs: [resolved issues](https://github.com/asciidoctor/asciidoctor/issues?q=label%3Av2.0.13+is%3Aclosed) | [full diff](https://github.com/asciidoctor/asciidoctor/compare/v2.0.12...v2.0.13) | [issues resolved in 2.0.x (cumulative)](https://github.com/asciidoctor/asciidoctor/milestone/33?closed=1) ## Changelog ### Bug Fixes * Don't allow counter and counter2 attribute directives to override locked attributes (#3939) * Fix crash when resolving next value in sequence for counter with non-numeric value (#3940) * Rollback change for #3470, which added logic to remove leading and trailing empty lines in an AsciiDoc include file; instead skip empty lines before processing document header (#3997) * Update default stylesheet to remove dash in front of cite on nested quote block (#3847) * Don't mangle formatting macros when uppercasing section titles in man page output (#3892) * Don't escape hyphen in manname in man page output * Remove extra .sp before content of verse block in man page output * Fix layout of footnotes in man page output (#3989) * Fix formatting of footnote text with URL in man page output (#3988) * Remove redundant trailing space on URL followed by non-adjacent text in man page output (#4004) * Use .bp macro at location of page break in man page output (#3992) ### Improvements * Extract method to create lexer and formatter in Rouge adapter (#3953) (*@Oblomov*) * Add support for pygments.rb 2.x (#3969) (*@slonopotamus*) * Allow NullLogger to be enabled by setting the `:logger` option to a falsy value (#3982) * Substitute attributes in manpurpose in NAME section of man page (#4000) * Output all mannames in name section of HTML output for manpage doctype (#3757) ### Build / Infrastructure * Import documentation for processor into this repository (#3861) (*@graphitefriction*) * Run tests against both pygments.rb 1.x and 2.x (#3969) (*@slonopotamus*) ### Documentation * add Belarusian translation of built-in attributes (PR #3928) (*@morganov*) ## Credits A very special thanks to all the **awesome** [supporters of the Asciidoctor OpenCollective campaign](https://opencollective.com/asciidoctor) who provided critical funding for the development of this release as well as ongoing development of the project. We'd also like to thank the maintainers of the Rouge project, who helped us work through the compatibility issue with Rouge and applied a fix to restore it ahead of our own release. 2021-04-10T22:52:09+00:00 uBlock 1.34.1b4 uBlock 1.34.1b4 2021-04-11T13:44:08+00:00 [Commits to master since this release](https://github.com/gorhill/uBlock/compare/1.34.1b4...master) To install the developer build: - **Firefox**: Click [uBlock0_1.34.1b4.firefox.signed.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1b4/uBlock0_1.34.1b4.firefox.signed.xpi). - [uBO works best on Firefox](https://github.com/gorhill/uBlock/wiki/uBlock-Origin-works-best-on-Firefox). - **Chromium**: Install from the Chrome Web Store (CWS): <https://chrome.google.com/webstore/detail/ublock-origin-dev-build/cgbcahbpdhpcegmbfconppldiemgcoii>. - **Thunderbird**: Download [uBlock0_1.34.1b4.thunderbird.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1b4/uBlock0_1.34.1b4.thunderbird.xpi), then drag-n-drop it into Thunderbird's _Add-ons Manager_ pane (Thunderbird 65+ required) --- ## Closed as fixed - [Block FLoC checks in Chrome/uBO](https://github.com/uBlockOrigin/uBlock-issues/issues/1553) - [json-prune doesn't remove all properties by full wildcard](https://github.com/uBlockOrigin/uBlock-issues/issues/1545) ## Notable commits without en entry in the issue tracker - [Fix regex used to extract attribute name in element picker](https://github.com/gorhill/uBlock/commit/9a94ba0a226f8d9589806e07409ee6de4925a721) - [Add support for removal of response headers](https://github.com/gorhill/uBlock/commit/f876b68171ff307f27601225607a6801f400437d) 2021-04-11T13:44:08+00:00 dnsdbq v2.5.4 dnsdbq v2.5.4 2021-04-11T15:05:27+00:00 Add DNSDBQ_CONFIG_FILE environment variable below which can specify a different configuration file to use 2021-04-11T15:05:27+00:00 radare2 5.2.0 radare2 5.2.0 2021-04-11T17:30:44+00:00 Release Notes ------------- Version: 5.2.0 Previous: 5.1.1 Commits: 316 Contributors: 35 <details><summary>Contributors</summary><p> [Alexandr](mailto:businesscode.rnr@gmail.com) [Alexandr](mailto:guzhsasha@gmail.com) [Alexis Ehret](mailto:08a@riseup.net) [Alucowie](mailto:anthoine.bourgeois@gmail.com) [Basstorm](mailto:37471354+Basstorm@users.noreply.github.com) [Dennis Goodlett](mailto:dennis+git_swoops@hurricanelabs.com) [Florian M"arkl](mailto:info@florianmaerkl.de) [Francesco Tamagni](mailto:mrmacete@protonmail.ch) [Khairul Azhar Kasmiran](mailto:kazarmy@gmail.com) [Lars Wrenger](mailto:lars@wrenger.net) [Murphy](mailto:dark.cancerbero@gmail.com) [Pamplemousse](mailto:xav.maso@gmail.com) [Paul I](mailto:pelijah@users.noreply.github.com) [RHL120](mailto:60626307+RHL120@users.noreply.github.com) [Reviakin Evgeny](mailto:reviakinea@gmail.com) [Roman Valls Guimera](mailto:brainstorm@users.noreply.github.com) [Sylvain Pelissier](mailto:sylvain.pelissier@gmail.com) [Taggggy](mailto:51325096+Taggggy@users.noreply.github.com) [condret](mailto:condr3t@protonmail.com) [el-goe](mailto:64834392+el-goe@users.noreply.github.com) [gogo](mailto:gogo246475@gmail.com) [gogo2464](mailto:57051187+gogo2464@users.noreply.github.com) [intruder-kat](mailto:37055173+intruder-kat@users.noreply.github.com) [ivan tkachenko](mailto:me@ratijas.tk) [meme](mailto:18178821+meme@users.noreply.github.com) [mio](mailto:ziqiaokong@outlook.com) [mrglm](mailto:mrglm@protonmail.com) [murphy](mailto:dark.cancerbero@gmail.com) [pancake](mailto:pancake@nopcode.org) [ramikg](mailto:72725910+ramikg@users.noreply.github.com) [soroosh-chabi](mailto:soroosh.chabi@gmail.com) [temp1337](mailto:temp1337@github) [valdaarhun](mailto:valdaarhun@github.com) [wargio](mailto:wargio@libero.it) </details> <details><summary>TLDR</summary><p> **anal** * Use =RS 8 for avr * Add =RS directive in reg profiles to define default value * Fix jump table analysis issue for r2ghidra.v850 (#18550) * Test for pcdelta ARM ldr fix * Fix pcdelta for ARM esil LDR * V850 jmptable fix, cmpval is almost always -1 and slows anal to a crawl (#18498) * Add missing v850 calling convention definition file * Remove unused type FcnTreeIter * Fix oobread ppc plugin * handleMidFlags: Reset ds->midflags on entry * asm.flags.middle: Don't split bb instruction * aae: Realign on fcn start if not in bb * Fix duplicate vtable entries after 'aaa' * af-*: Remove function flags too * Fix cX command and minor cleanup * Fix PSW register bits definition for v850 * Add support for jump tables on v850 * Fix #18284 (json command returning empty string) * Cd1 Cd2, Cd4, Cd8 are aliases for Cd<space>[1248] * Fix SN register value for linux-arm64 * Improve reg profile parsing and error handling * Detect shift for the first switch case * * Fix gcc 9.2.0 kind of x86_64 jumptables * New command: aaw, flag all words pointing to known flags **android** * Add r_file_binsh() and avoid hardcoding /bin/sh for Termux **api** * Remove some exit() calls in libr * Rename r_cons_memcat to r_cons_write * ABI/API break. RAnnotationCode->RCodeMeta * Use more r_str_ncpy and improve it to not alloc beyond nullbyte * Add r_vector_flush() * Apply desired lifecycle of esil syscall and interrupt handlers in esil_dummy plugin * Add r_anal_esil_{syscall/interrupt}_{get/del} **bin** * Parse the symbols from the ELF .gnu_debugdata section * Support x86_64 and macOS dyld caches (#18570) * Fix wrong demangling of tiff swab16 bit data * Use r_bin_import_free() as <free> cb for imports list (#893) * Improve python disassembler and binary parser * Fix large loading times when parsing encrypted/fuzzed macho * Do not consider ELFs with .gnu_debugdata section as stripped * Fix Mach-O related coverity issues * Fix coverity issues in xnu kernelcache * Fix ASAN crash when allocating more relocs than the filesize * Fix long time analyzing oob objc data * * Fix long time analyzing oob objc data * Fix asan crash found in r2_hoobr_dex_loadcode * Fix oobread bug in r_str_(ndup|nlen) APIs spotted by ASAN in SMD parser * Fix UAF in rz_bin_reset_strings() * Fix assert in iSj for invalid size sections * Minor ELF cleanup, using more size_t and ut64.max instead of 0 * Add Support for new CoreSymbolication element format * Unset io.cache when not needed after bin.cache * Fix warning message when loading files with relocs **build** * Move shlr/tcc into libr/parse/c * Fix meson build issues related to grub (#18554) * Fix the failure in finding the executing user's ID during install (#18508) * Add the nogpl meson option * Fix version not being updated after running sys/termux.sh * Update doc/license for more clarifications * Add plugins=nogrub option for meson * Improve sys/termux.sh checks * Do not run sys/ldconfig on Android * Add missing use_fork and use_dylink to meson * Fix #18397 - Be less strict when running sys/install.sh as root * Fix debian32 in CI * Allow custom CFLAGS for Debian packaging * Add meson -Dplugins=a,b,c to build only the specified plugins * Fixes to make the r2blob shine again * Add 32bit Debian packaging and bonus CI fixes * Add use_ssl meson option to be in sync with acr behaviour **charset** * Implement ps, psz, psj and psj with charset support * Support multi-byte input in charset * Add more runes to pokered **ci** * Fix linux-static pub action **cmd** * Sync om and omj output **cons** * Fix Ctrl-J issue and remove redundant code in 'Ctrl-J' block * Implement RConsPixel and RBraile APIs * Fix #16254 - grep expression parse improvement * Fix null derefs on RCons when no context is provided **core** * Fix #18412 - Add R2_IGNVER variable to load plugins ignoring the version * Remove asm.bb (asm.bb.line -> asm.lines.bb, asm.bb.middle -> asm.bbmiddle) **crypto** * Update to use keys that can be programmed onto a CPS2 **debug** * Implement drcq and show it in visual debug/emu * Fix #18502 - dangling pointers issues in dbm * Revert "dmi commands handle symbols, exports, main, entries too * dmi commands handle symbols, exports, main, entries too * Implement dmis command as an alias for .dmi* * Workaround the dmi issue by using rabin2 in macOS for now * Add dbg.maxsnapsize to avoid snapping huge maps **debug"** * Revert "dmi commands handle symbols, exports, main, entries too **decompiler** * Detect retdec decompiler (pdz) in cmd.pdc **diff** * Add abstract Levenshtein dist * Abstract r_diff_levenshtein_path * Add Levenshtiend path API to **disasm** * Honor asm.cpu for asm.arch=ppc.gnu * Fix #18511 - Add dwarf info in pdj * Add the m68k.gnu disassembler plugin * Show overlapped flags if requested and show them differently (#706) * Honor cfg.debug in asm.section using dmi. * Improve asm.meta=false for 16, 32 and 64 words * Fix #17761 - Do not trim the "ptr " when asm.syntax=masm * Add pi+ and pi- commands as aliases for 'pi +' and 'pi -' * Fix asm.lines.bb with asm.sections set **dwarf** * Implement CLj command and improve CL output **emu** * Apply desired lifecycle of esil syscall and interrupt handlers in esil_dummy plugin * Add dummy interrupt and syscall handlers **esil** * Add r_anal_esil_{syscall/interrupt}_{get/del} * Boolify all the ESIL callbacks * Fix some asserts and nullderef spotted on arm64 * Esil plugin management APIs * * Add r_anal_esil_{get/del}_op * * Fix deactivating plugins on r_anal_esil_free * * Add local getter for active esil plugins **fs** * Fix absolute paths and add r_return guards * Fix fs.posix.cat and use r_sys_dir instead of reimplement (-48LOC) **graph** * Add support for highlighted edges in graphviz * Add ageh command to let users define which node links should be highlighted **hash** * Implement ssdeep fuzzy hashing algorithm * Fix name collision for SHA functions **io** * Close #18257 / Remove RIODescData * Enable io-plugins to set bin.baddr on launch * Fix comment * Implement custom bit size cyclic memory layout and wrap flags in * Rename r_io_map_get_for_fd to r_io_map_get_by_fd * Rename r_io_map_resolve to r_io_map_get * Rename r_io_map_get to r_io_map_get_at * Implement slurp:// uri handler plugin * Implement omd command, as a simplified version of om * Fix infinite loop in r_io_map_next_available * Fix #17049 - oa whithout filename specify, add oa test * Add help message for winkd://? and improve desc **json** * iVj must print valid json, not empty output (#18571) * Add json version output to `r2 -vj` * rasm2 -L now shows an array of objects * Fix #18284 (json command returning empty string) **lang** * Add #!*? command to show rlang plugin examples * Add support to the V programming language for scripting **print** * Implement pFX command exemplifying the use of r_sys_unxz() * Android Binary XML support (#18545) * Improved pdc, added pdco and pdcj, print orphan nodes * Fix alignment issues in RTable with utf8 fields **reg** * Fix null printf issue in arpi command **search** * Import @siguza's arm64 xref finder * Show results after pressing ^C in /ad * Fix /ad of multiple consecutive instructions **shell** * Fix proper hash comments & quotes mix (#18551) * Implement whoami and uid commands * rasm2 -Lj works the same way as -jL * Add variable autocompletion class in !!! * Add scr.loopnl to add a newline on all the @@ loops **test** * Add R2R_SKIP_ASM env var handling in r2r **tools** * Fix #18391 - Show help in r2pm even before initializing the db **util** * Add r_file_find to recursive list files and subdirectories into a list * Rename dep in rbtree.c to depth * Rename d, d2, d3 in rbtree.c to direction ... * Optimize r_rbtree_cont_{first/last} #18485 * Implement r_rbtree_cont_node_prev (siol eternal) * Implement r_rbtree_cont_last * Fix endless loop in r_rbtree_cont_node_next * Add unit test for r_rbtree_cont_node_next * Fix segfaults in rbtree.c (parent backlink) * Add r_rb_cont_tree_node_next (SIOL Eternal) * Add backlink to parent in RBNode (SIOL Eternal) * Add r_rb_cont_tree_find_node (SIOL Eternal) **visual** * Add agfb for braile graphs * Revert "Fix memleaks in agraph by keeping a separated list of dummy nodes * Fix memleaks in agraph by keeping a separated list of dummy nodes * Initial implementation of graph edge highlighting **visual"** * Revert "Fix memleaks in agraph by keeping a separated list of dummy nodes </p></details> Associated releases: * [r2ghidra](https://github.com/radareorg/r2ghidra) (decompiler plugin) * [iaito](https://github.com/radareorg/iaito) (Qt UI) * [radare2-extras](https://github.com/radareorg/radare2-extras) (bonus plugins) * [radare2-bindings](https://github.com/radareorg/radare2-bindings) (python, node, ...) See below for the changelogs: <details><summary>Highlights</summary><p> ### Improved V850 Support This release comes with several improvements for the V850 (in concret e2) cpu. The default disassembler is not really supporting many of the instructions of this architecture, but you can use `r2ghidra` as disassembler and analyzer which comes with a bunch of improvements for this architecture. Here's a list of the most relevant changes: * Add support for cyclic memory layouts, this memory model is only supported in some rare architectures like s390, VAX, old ARMs and some V850. And this is not supported by any other static analysis tool (only by some closed source emulators from what @trufae could find). * Implemented jump table analysis and pointer table size prediction, requires latest r2ghidra to work * Add /cu command to find UDS dispatch tables and pointers for CAN BUS handling firmwares * The new `omd` command makes creating memory layouts for embedded firmwares much easier, but at the end it's just a simplified version of `om`. * Add `slurp://` uri handler which does the same as `oom` to reload the given disk file into a malloc:// which is ideal for loading firmware dumps with ram contents for proper emulation. * Fix PSW register bits definition for v850 * Added default calling convention for v850e2-gcc ## ARM64/Apple support * Android binary XML decoder available in the new `pFa` command (thanks @meme) * Added support for the new CoreSymbolication file format (thanks @mrmacete ) * Support the latest dyldcaches from macOS/arm64 (thanks @meme) * Add core plugin using @Siguza 's arm64xref search code which is 200x faster than /r with capstone * Fixed large loading times and segfaults when loading some corrupted mach0 binaries * Improved ARM32 LDR esil emulation which may fix some missing xrefs ## Analysis * New command: aaw, flag all words pointing to known flags * Float/Double/LongDouble types supported in `pf [fFG]` and `wv[fFG]` * Better handling errors when parsing invalid register profiles * Add `=RS` directive to define the default register size. * Assign `pdc, pdd, pdg, pdz` to their respective decompilers * Honor all asm.cpu values for the powerpc GNU disassembler * Added levenstein binary diffing path APIs * Improve `pdc` pseudo-decompiler output (show orphaned bbs) + pdcj support * Add support for new crypto keys on CPS2 ## Search * Add brand new implementation of ssdeep (by @trufae) under the MIT license. * Fix `/ad` command results are now correct and tested. * New `sixref` command (see siguza's plugin in the arm64 section) ## Shell * Add `lsr`, `whoami` and `uid` commands * Move the Vlang rlang plugin from -extras to core. it's now stable and ready to use. * Add scr.loopnl to add a newline on all the @@ loops * Improved `dmi` commands to load more symbols in debugger mode * Fix #16254 - grep expression parse improvement ##cons * Add freebsd-x86-32 support for sflib based shellcodes with `ragg2` ## Binaries * Added bin.cache as a high level value for io.cache * Add support for loading symbols from `.gnu_debugdata` sections on ELF binaries * Support latest dyldcache and coresymbolication files for macos-arm64 and ios * Fixed segfaults and large loading times on 6502, PYC, DEX, MACHO, ELF, DWARF formats * Dwarf information is now listed in the output of `pdj` and other `CL` listings ## Visual * Add support for highlighting edges in graphviz and ascii graphs * Honor cfg.charset in `w`, `ps`, `psz` and `psj` (thanks @gogo2464 ) * Add minigraph support in visual (check `graph.mini`) * Support multibyte input conversions in custom charset encodings * Improved glob expression text filtering (thanks @as0ler ) * Implemented braile-art graph rendering (`afbg`) * Show status register bits in visual debugger mode ## Build * Clarify license usage in target build for core modules and installed plugins ( * Fix static builds with meson * Handle use_ssl, use_fork, use_dylib, nogrub, nogpl and custom plugin builds with meson * Cleaned up slow tests and make CI run in GHA in 8min * Import patches from Termux </p></details> 2021-04-11T17:30:44+00:00 seaweedfs 2.39 seaweedfs 2.39 2021-04-12T02:48:09+00:00 * Add `weed gateway` for easier submitting blobs * FUSE Mount * Release view cache to avoid OOM * Filer * Avoid stuck uploader #1980 * Add a basic FTP server in Java #1968 2021-04-12T02:48:09+00:00 netdata v1.30.1 netdata v1.30.1 2021-04-12T13:19:51+00:00 This is a patch release to address discovered issues since 1.30.0. ## Acknowledgments - [@jsoref](https://github.com/jsoref) for fixing numerous spelling mistakes. ## Documentation - Fix grammar in ACLK README.md. ([#10898](https://github.com/netdata/netdata/pull/10898), [@slimanio](https://github.com/slimanio)) - Update news and GIF in README, fix typo. ([#10900](https://github.com/netdata/netdata/pull/10900), [@joelhans](https://github.com/joelhans)) - Fix spelling mistakes in various places. ([#10428](https://github.com/netdata/netdata/pull/10428), [@jsoref](https://github.com/jsoref)) ## Packaging / Installation - Don’t use glob expansion in argument to `cd` in updater. ([#10936](https://github.com/netdata/netdata/pull/10936), [@Ferroin](https://github.com/Ferroin)) - Bumped version of OpenSSL bundled in static builds to 1.1.1k. ([#10884](https://github.com/netdata/netdata/pull/10884), [@Ferroin](https://github.com/Ferroin)) - Fix bundling of ACLK-NG components in dist tarballs. ([#10894](https://github.com/netdata/netdata/pull/10894), [@Ferroin](https://github.com/Ferroin)) ## Bug Fixes - Fix memory corruption issue when executing context queries in RAM/SAVE memory mode. ([#10933](https://github.com/netdata/netdata/pull/10933), [@stelfrag](https://github.com/stelfrag)) - Add a CRASH event when the agent fails to properly shutdown. ([#10893](https://github.com/netdata/netdata/pull/10893), [@stelfrag](https://github.com/stelfrag)) - Fix incorrect health log entries. ([#10822](https://github.com/netdata/netdata/pull/10822), [@stelfrag](https://github.com/stelfrag)) 2021-04-12T13:19:51+00:00 TheHive 4.1.3 TheHive 4.1.3 2021-04-12T16:25:05+00:00 **Implemented enhancements:** - [Improvement] Cleanup deprecated filter usage [\#1922](https://github.com/TheHive-Project/TheHive/issues/1922) - [Improvement] Make the property "Imported" in alerts optimised for index [\#1923](https://github.com/TheHive-Project/TheHive/issues/1923) - [Feature Request] Display case templates in alphabetic order in "New case" menu [\#1925](https://github.com/TheHive-Project/TheHive/issues/1925) - [Enhancement] Prevent the application to start if database initialisation fails [\#1935](https://github.com/TheHive-Project/TheHive/issues/1935) - [Enhancement] Improve performance [\#1946](https://github.com/TheHive-Project/TheHive/issues/1946) - [Enhancement] Remove blocking queries in some UI pages [\#1948](https://github.com/TheHive-Project/TheHive/issues/1948) - [Enhancement] Use polluingDuration config from the UI [\#1951](https://github.com/TheHive-Project/TheHive/issues/1951) - [Enhancement] Disable confirm buttons in import dialogs [\#1953](https://github.com/TheHive-Project/TheHive/issues/1953) - [Enhancement] Add environment file in service [\#1954](https://github.com/TheHive-Project/TheHive/issues/1954) **Fixed bugs:** - [Bug] Add "Not assigned" to Assignee field on task page for tasks without an assigned user [\#1508](https://github.com/TheHive-Project/TheHive/issues/1508) - [Bug] (Still) slow loading of list-tags endpoint with 4.1.2 [\#1914](https://github.com/TheHive-Project/TheHive/issues/1914) - [Bug] Aggregation on custom fields provides incorect result [\#1921](https://github.com/TheHive-Project/TheHive/issues/1921) - [Bug] Very slow load of Case Task list in UI in 4.1.2 [\#1927](https://github.com/TheHive-Project/TheHive/issues/1927) - [Bug] Task "Take" Button not working [\#1931](https://github.com/TheHive-Project/TheHive/issues/1931) - [Bug] Cluster: new nodes fail to start when the oldest node has been restarted [\#1934](https://github.com/TheHive-Project/TheHive/issues/1934) - [Bug] Index status page is very slow [\#1936](https://github.com/TheHive-Project/TheHive/issues/1936) - [Bug] Update of color in tags [\#1950](https://github.com/TheHive-Project/TheHive/issues/1950) 2021-04-12T16:25:05+00:00 netbox v2.10.9 netbox v2.10.9 2021-04-12T17:29:14+00:00 ## Enhancements * [#5526](https://github.com/netbox-community/netbox/issues/5526) - Add MAC address search field to VM interfaces list * [#5756](https://github.com/netbox-community/netbox/issues/5756) - Omit child devices from non-racked devices list under rack view * [#5840](https://github.com/netbox-community/netbox/issues/5840) - Add column to cable termination objects to display cable color * [#6054](https://github.com/netbox-community/netbox/issues/6054) - Display NAPALM-enabled device tabs only when relevant * [#6083](https://github.com/netbox-community/netbox/issues/6083) - Support disabling TLS certificate validation for Redis ## Bug Fixes * [#5805](https://github.com/netbox-community/netbox/issues/5805) - Fix missing custom field filters for cables, rack reservations * [#6070](https://github.com/netbox-community/netbox/issues/6070) - Add missing `count_ipaddresses` attribute to VMInterface serializer * [#6073](https://github.com/netbox-community/netbox/issues/6073) - Permit users to manage their own REST API tokens without needing explicit permission * [#6081](https://github.com/netbox-community/netbox/issues/6081) - Fix interface connections REST API endpoint * [#6082](https://github.com/netbox-community/netbox/issues/6082) - Support colons in webhook header values * [#6108](https://github.com/netbox-community/netbox/issues/6108) - Do not infer tenant assignment from parent objects for prefixes, IP addresses * [#6117](https://github.com/netbox-community/netbox/issues/6117) - Handle exception when attempting to assign an MPTT-enabled model as its own parent * [#6131](https://github.com/netbox-community/netbox/issues/6131) - Correct handling of boolean fields when cloning objects 2021-04-12T17:29:14+00:00 pdfx v1.4.1 pdfx v1.4.1 2021-04-12T19:06:33+00:00 https://github.com/metachris/pdfx/compare/v1.3.1...v1.4.1 2021-04-12T19:06:33+00:00 python_api v2.2.0 python_api v2.2.0 2021-04-13T00:38:53+00:00 New object analyzer module to simplify hostname and IP address analysis for the most common security and research use cases. Updated documentation and a new README file. 2021-04-13T00:38:53+00:00 turbinia 20210413 turbinia 20210413 2021-04-13T08:28:41+00:00 * plaso unattended fix * metrics dashboards 2021-04-13T08:28:41+00:00 syncthing v1.16.0-rc.1 syncthing v1.16.0-rc.1 2021-04-13T08:41:24+00:00 This release adds untrusted / encrypted devices for public testing. It should still be considered beta / testing-only. Bugfixes: - #7026: panic: key not found when repairing DB - #7533: Remote completion on untrusted devices is incorrect - #7548: Upgrading syncthing-relaysrv to 1.15.0 fails in post-installation - #7557: Encrypted folder(s) on Windows reported as "Unexpected Objects" Enhancements: - #109: Support for file encryption (e.g. non-trusted servers) - #3322: GUI "Out of sync" modal uses massive amounts of CPU in browser - #7520: Ask for confirmation before reverting/overriding a folder 2021-04-13T08:41:24+00:00 uBlock 1.34.1b5 uBlock 1.34.1b5 2021-04-13T14:29:27+00:00 [Commits to master since this release](https://github.com/gorhill/uBlock/compare/1.34.1b5...master) To install the developer build: - **Firefox**: Click [uBlock0_1.34.1b5.firefox.signed.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1b5/uBlock0_1.34.1b5.firefox.signed.xpi). - [uBO works best on Firefox](https://github.com/gorhill/uBlock/wiki/uBlock-Origin-works-best-on-Firefox). - **Chromium**: Install from the Chrome Web Store (CWS): <https://chrome.google.com/webstore/detail/ublock-origin-dev-build/cgbcahbpdhpcegmbfconppldiemgcoii>. - **Thunderbird**: Download [uBlock0_1.34.1b5.thunderbird.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1b5/uBlock0_1.34.1b5.thunderbird.xpi), then drag-n-drop it into Thunderbird's _Add-ons Manager_ pane (Thunderbird 65+ required) --- ## Closed as fixed - [Block FLoC checks in Chrome/uBO](https://github.com/uBlockOrigin/uBlock-issues/issues/1553) - [json-prune doesn't remove all properties by full wildcard](https://github.com/uBlockOrigin/uBlock-issues/issues/1545) ## Notable commits without en entry in the issue tracker - [Finalize 3rd-party scripts/frames mini-filtering widget](https://github.com/gorhill/uBlock/commit/58505cfddcabf216fdfdd1150eedfc2f28f36435) - [Remove advanced setting `ignoreRedirectFilters`](https://github.com/gorhill/uBlock/commit/f5b453fae3c77363367cdf9494e90fe276042a72) - [Fix regex used to extract attribute name in element picker](https://github.com/gorhill/uBlock/commit/9a94ba0a226f8d9589806e07409ee6de4925a721) - [Add support for removal of response headers](https://github.com/gorhill/uBlock/commit/f876b68171ff307f27601225607a6801f400437d) 2021-04-13T14:29:27+00:00 node_redis v3.1.1 node_redis v3.1.1 2021-04-13T16:49:14+00:00 # Enhancements * Fix a potential exponential regex in monitor mode (#1578) 2021-04-13T16:49:14+00:00 IDAtropy python2 IDAtropy python2 2021-04-14T08:23:45+00:00 This is the last release supporting python 2.X and IDA versions 6.X to IDA 7.3. 2021-04-14T08:23:45+00:00 igraph 0.9.2 igraph 0.9.2 2021-04-14T11:36:02+00:00 ### Added - CMake package files are now installed with igraph. This allows `find_package(igraph)` to find igraph and detect the appropriate compilation options for projects that link to it. ### Fixed - igraph can now be used as a CMake subproject in other CMake-based projects. - The documentaton can now be built from the release tarball. - Configuration will no longer fail when the release tarball is extracted into a subdirectory of an unrelated git repository. - The generated pkg-config file was incorrect when `CMAKE_INSTALL_<dir>` variables were absolute paths. - On Unix-like systems, the library name is now `libigraph.so.0.0.0`, as it used to be for igraph 0.8 and earlier. - Fixed a return type mismatch in parser sources, and fixed some warnings with recent versions of gcc. - Fixed a bug in `igraph_get_shortest_paths_dijkstra()` and `igraph_get_shortest_paths_bellman_ford()` that returned incorrect results for unreachable vertices. ### Other - Improved installation instructions and tutorial. 2021-04-14T11:36:02+00:00 snoopy snoopy-2.4.13 snoopy snoopy-2.4.13 2021-04-14T12:31:44+00:00 2021-04-14T12:31:44+00:00 uBlock 1.34.1rc0 uBlock 1.34.1rc0 2021-04-14T15:06:48+00:00 [Commits to master since this release](https://github.com/gorhill/uBlock/compare/1.34.1rc0...master) To install the developer build: - **Firefox**: Click [uBlock0_1.34.1rc0.firefox.signed.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1rc0/uBlock0_1.34.1rc0.firefox.signed.xpi). - [uBO works best on Firefox](https://github.com/gorhill/uBlock/wiki/uBlock-Origin-works-best-on-Firefox). - **Chromium**: Install from the Chrome Web Store (CWS): <https://chrome.google.com/webstore/detail/ublock-origin-dev-build/cgbcahbpdhpcegmbfconppldiemgcoii>. - **Thunderbird**: Download [uBlock0_1.34.1rc0.thunderbird.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1rc0/uBlock0_1.34.1rc0.thunderbird.xpi), then drag-n-drop it into Thunderbird's _Add-ons Manager_ pane (Thunderbird 65+ required) --- ## Closed as fixed - [Block FLoC checks in Chrome/uBO](https://github.com/uBlockOrigin/uBlock-issues/issues/1553) - [json-prune doesn't remove all properties by full wildcard](https://github.com/uBlockOrigin/uBlock-issues/issues/1545) ## Notable commits without en entry in the issue tracker - [Remove advanced setting `ignoreScriptInjectFilters`](https://github.com/gorhill/uBlock/commit/bc8c1d00ad728e854b3caf35231d808515058d3f) - [Finalize 3rd-party scripts/frames mini-filtering widget](https://github.com/gorhill/uBlock/commit/58505cfddcabf216fdfdd1150eedfc2f28f36435) - [Remove advanced setting `ignoreRedirectFilters`](https://github.com/gorhill/uBlock/commit/f5b453fae3c77363367cdf9494e90fe276042a72) - [Fix regex used to extract attribute name in element picker](https://github.com/gorhill/uBlock/commit/9a94ba0a226f8d9589806e07409ee6de4925a721) - [Add support for removal of response headers](https://github.com/gorhill/uBlock/commit/f876b68171ff307f27601225607a6801f400437d) 2021-04-14T15:06:48+00:00 PacketFence v10.3.0 PacketFence v10.3.0 2021-04-14T18:36:03+00:00 ### New Features * Static routes management via admin gui * Aruba CX support * Aruba 2930M Web Authentication and Dynamic ACL support (#6158) * Meraki DPSK support * Ruckus DPSK support * Support for Ruckus SmartZone MAC authentication in non-proxy modes (#6201) * Bluesocket support (#5878) * Support for SCEP in `pfpki` (#6213) ### Enhancements * Improved the failover mechanisms when an Active Directory or LDAP server is detected as dead * Expiration of the local accounts created on the portal can now be set on the source level * pfacct and radiusd-acct can now both be enabled together (radiusd-acct proxies to pfacct) * Added CoA support to Aerohive module * Added role based enforcement (Filter-Id) support to Extreme module * Use Called-Station-SSID attribute as the SSID when possible * Added CLI login support to Huawei switch template * Added detectionBypass in DNS resolver (#6028) * Improve support of Android Agent for EAP-TLS and EAP-PEAP * Improve CLI login support on HP and Aruba switches * Use the "Authorization" header when performing API calls to Github in the OAuth context * Replace xsltproc/fop by asciidoctor-pdf (#5968) * FortiGate Role Based Enforcement (#5645) * Add support for roles (RBAC) for Ruckus WLAN controllers (#2530) * Upgrade to go version 1.15 (#6044) * Build ready-to-use Vagrant images for integration tests and send them to Vagrant cloud (#6099) * Documentation to configure Security Onion 2.3.10 * Added integration tests for 802.1X wireless and wireless MAC authentication (#6114) * Restrict create, update, and delete operations to the default and global tenant users (#6075) * Remove pftest MySQL tuner (#6130) * Allow Netflow address to be configured (#6139) * Deprecated fencing whitelist * Description field for L2 and routed networks (#5829) * Updated Stripe integration to use Stripe Elements (API v3) (#6121) * Added Cisco WLC 9800 configuration documentation * Inheritance on parent role on Role and Web Auth * Enhance CLI login on SG300 switches * Enable/disable the natting traffic for inline networks * Remove unused table userlog (#6170) * Clarifications on Ruckus Role-by-Role capabilities (#6201) * DNS/IP attributes in pfpki certificates (#6213) * Additional template attributes in certificate profile (#6213) * Remove unused table inline_accounting (#6171) * Make pfdhcplistener tenant aware (#6204) * Upgrade to MariaDB 10.2.37 (#6149) ### Bug Fixes * Switch defined by MAC address are not processed by pfacct in cluster mode (#5969) * Restart switchport return TRUE if MAC address is not found in locationlog for bouncePortCoA (#6013) * Switch template: CLI authorize attributes ignored (#6009) * ubiquiti_ap_mac_to_ip task doesn't update expires_at column in chi_cache table (#6004) * A switch can't override switch group values using default switch group values (#5998) * web admin: timer_expire and ocsp_timeout are not displayed correctly (#5961) * web admin: Realm can't be selected as a filter on a connection profile (#5959) * API: remove a source doesn't remove rules from authentication.conf (#5958) * web admin: high-availability setting is not display correctly when editing an interface (#5963) * SSIDs are not hidden by default when creating a provisioner (#5952) * with_aup is correctly displayed on GUI (#5954) * web admin: sender is wrong when you use Preview feature (#6023) * sponsor guest registration: unexpected strings in email subject (#3669) * Use the proper attribute name for Mikrotik in returnRadiusAccessAccept (#6051) * Audit log: profile has an empty value when doing Ethernet/Wireless-NoEAP (#5977) * pfacct stores 00:00:00:00:00:00 MAC in DB when Calling-Station-ID is XXXX-XXXX-XXXX (#6109) * Update the location log when the Called-Station-Id changes (#6045) * Only enable NetFlow in iptables if NetFlow is enabled (#6080) * Firewall SSO: take username from accounting data if available in place of database (#6148) 2021-04-14T18:36:03+00:00 python_api v2.3.0 python_api v2.3.0 2021-04-14T21:32:04+00:00 Adds support for RiskIQ Illuminate API, specifically Reputation Score, plus a few bug fixes and documentation improvements. 2021-04-14T21:32:04+00:00 syncthing v1.16.0-rc.2 syncthing v1.16.0-rc.2 2021-04-15T10:08:54+00:00 This release adds untrusted / encrypted devices for public testing. It should still be considered beta / testing-only. Bugfixes: - #7026: panic: key not found when repairing DB - #7533: Remote completion on untrusted devices is incorrect - #7548: Upgrading syncthing-relaysrv to 1.15.0 fails in post-installation - #7551: QUIC: failed to determine receive buffer size: doesn't have a SyscallConn - #7557: Encrypted folder(s) on Windows reported as "Unexpected Objects" - #7568: panic: runtime error: slice bounds out of range [24:3] Enhancements: - #109: Support for file encryption (e.g. non-trusted servers) - #3322: GUI "Out of sync" modal uses massive amounts of CPU in browser - #7520: Ask for confirmation before reverting/overriding a folder 2021-04-15T10:08:54+00:00 osquery 0.0.2 osquery 0.0.2 2021-04-15T11:48:17+00:00 2021-04-15T11:48:17+00:00 mattermost-server v5.34.0 mattermost-server v5.34.0 2021-04-15T12:54:45+00:00 Mattermost Platform Release v5.34.0 2021-04-15T12:54:45+00:00 TheHive 4.1.4 TheHive 4.1.4 2021-04-15T15:36:53+00:00 **Implemented enhancements:** - [Feature Request] Sort case templates in alert Import drop down menu [\#1956](https://github.com/TheHive-Project/TheHive/issues/1956) - [Enhancement] Make alert deletion more robust [\#1967](https://github.com/TheHive-Project/TheHive/issues/1967) **Fixed bugs:** - [Bug] Alert counter refresh not working [\#1911](https://github.com/TheHive-Project/TheHive/issues/1911) - [Bug] Enabled or Disabled Taxonomies doesn't work [\#1957](https://github.com/TheHive-Project/TheHive/issues/1957) - [Bug] TheHive 4.1.3-1 Task in Case is not visible [\#1964](https://github.com/TheHive-Project/TheHive/issues/1964) - [Bug] Aggregation on custom fields provides incorect result (again) [\#1965](https://github.com/TheHive-Project/TheHive/issues/1965) - [Bug] TheHive startup times out if schema evolution is long [\#1966](https://github.com/TheHive-Project/TheHive/issues/1966) - [Bug] Default analyst rol cant add TTPs [\#1968](https://github.com/TheHive-Project/TheHive/issues/1968) - [Bug] Links section should not be empty in Observables details view [\#1972](https://github.com/TheHive-Project/TheHive/issues/1972) - [Bug] Deleting a shared rule case for org2 - deleting observables (sharing to org2) from the org1. [\#1973](https://github.com/TheHive-Project/TheHive/issues/1973) - [Bug] AddTagToArtifact operation not working in 4.1.3 [\#1974](https://github.com/TheHive-Project/TheHive/issues/1974) 2021-04-15T15:36:53+00:00 mattermost-server v5.34.1 mattermost-server v5.34.1 2021-04-15T19:10:07+00:00 Mattermost Platform Release v5.34.1 2021-04-15T19:10:07+00:00 netbox v2.10.10 netbox v2.10.10 2021-04-15T19:35:04+00:00 ## Enhancements * [#5796](https://github.com/netbox-community/netbox/issues/5796) - Add DC terminal power port, outlet types * [#5980](https://github.com/netbox-community/netbox/issues/5980) - Add Saf-D-Grid power port, outlet types * [#6157](https://github.com/netbox-community/netbox/issues/6157) - Support Markdown rendering for report logs * [#6160](https://github.com/netbox-community/netbox/issues/6160) - Add F connector port type * [#6168](https://github.com/netbox-community/netbox/issues/6168) - Add SFP56 50GE interface type ## Bug Fixes * [#5419](https://github.com/netbox-community/netbox/issues/5419) - Update parent device/VM when deleting a primary IP * [#5643](https://github.com/netbox-community/netbox/issues/5643) - Fix VLAN assignment when editing VM interfaces in bulk * [#5652](https://github.com/netbox-community/netbox/issues/5652) - Update object data when renaming a custom field * [#6056](https://github.com/netbox-community/netbox/issues/6056) - Optimize change log cleanup * [#6144](https://github.com/netbox-community/netbox/issues/6144) - Fix MAC address field display in VM interfaces search form * [#6152](https://github.com/netbox-community/netbox/issues/6152) - Fix custom field filtering for cables, virtual chassis * [#6162](https://github.com/netbox-community/netbox/issues/6162) - Fix choice field filters (multiple models) 2021-04-15T19:35:04+00:00 openlibrary deploy-2021-04-15 openlibrary deploy-2021-04-15 2021-04-15T20:57:47+00:00 General: - @Yashs911 A11y: Fix Library Explorer Image Accessibility (#4958) - @shrey27tri01 Fix: Chart hover labels not visible on subjects/publisher pages (#4976) - @lephemere A11y: Library Explorer bookshelf keyboard focus fix (#5036) Dev: - @cdrini Remove makei18n from deploy.sh (#5008) - @cclauss fix: restart_servers.sh for docker-compose --profile and $HOSTNAME (#5009) - @cclauss Fix split_dump() to read in str, not bytes (#5006) - @cclauss Add @dhruvmanila to dependabot.yml Python (#5010) - @dhruvmanila Refactor: Move necessary dependencies to requirements_common.txt (#5021) - @cdrini Make oldev entrypoint absolute path (#5043) - @cdrini Make delete queries solr8 compatible (#4731) - @cdrini Make solrupdater handle works without titles (#4739) - @cdrini Fix solrbuilder memory issues (#4937) - @dhruvmanila refactor: Rename Python requirements* files (#5030) - @dhruvmanila feat: Add cache to GitHub actions (#5019) - @cdrini Remove venv in python_tests (#5053) - @cclauss Remove pystatsd in favor of statsd (#5018) - @cclauss Define repr(Nothing()) and str(Nothing()) (#5020) - @cdrini Fix lint-diff in GHA and fix CI var for make lint (#5049) Dependencies: - @cclauss Update from Python 3.9.2 to 3.9.4 (#4990) - @dependabot[bot] Bump pymarc from 4.0.0 to 4.1.0 (#4985) - @dependabot[bot] Bump pillow from 8.1.2 to 8.2.0 (#4983) - @dependabot[bot] Bump internetarchive from 1.9.9 to 2.0.2 (#5024) Stats: - PR Authors: @cdrini (7), @cclauss (6), @dhruvmanila (3), @Yashs911 (1), @lephemere (1), @shrey27tri01 (1) - PR Assignees: @cdrini (9), @cclauss (6), @mekarpeles (3), @bpmcneilly (1) Full diff: https://github.com/internetarchive/openlibrary/compare/deploy-2021-04-08...deploy-2021-04-15 PRs: [is:pr is:merged merged:2021-04-08T14:44:42Z..2021-04-15T20:52:32Z sort:updated-asc](https://github.com/internetarchive/openlibrary/pulls?q=is%3Apr%20is%3Amerged%20merged%3A2021-04-08T14%3A44%3A42Z..2021-04-15T20%3A52%3A32Z%20sort%3Aupdated-asc) 2021-04-15T20:57:47+00:00 binskim v1.7.5 binskim v1.7.5 2021-04-15T21:18:37+00:00 ## **v1.7.5** [NuGet Package](https://www.nuget.org/packages/Microsoft.CodeAnalysis.BinSkim/1.7.5) * BUGFIX: Fix import/export config using JSON file. [#349](https://github.com/microsoft/binskim/pull/349) * FEATURE: Add compiler report rule BA4001, which is disabled by default. [#350](https://github.com/microsoft/binskim/pull/350) * FEATURE: Add support to specific rule documentation in `HelpUri`. [#348](https://github.com/microsoft/binskim/pull/348) 2021-04-15T21:18:37+00:00 uBlock 1.34.1rc1 uBlock 1.34.1rc1 2021-04-16T13:03:59+00:00 [Commits to master since this release](https://github.com/gorhill/uBlock/compare/1.34.1rc1...master) To install the developer build: - **Firefox**: Click [uBlock0_1.34.1rc1.firefox.signed.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1rc1/uBlock0_1.34.1rc1.firefox.signed.xpi). - [uBO works best on Firefox](https://github.com/gorhill/uBlock/wiki/uBlock-Origin-works-best-on-Firefox). - **Chromium**: Install from the Chrome Web Store (CWS): <https://chrome.google.com/webstore/detail/ublock-origin-dev-build/cgbcahbpdhpcegmbfconppldiemgcoii>. - **Thunderbird**: Download [uBlock0_1.34.1rc1.thunderbird.xpi](https://github.com/gorhill/uBlock/releases/download/1.34.1rc1/uBlock0_1.34.1rc1.thunderbird.xpi), then drag-n-drop it into Thunderbird's _Add-ons Manager_ pane (Thunderbird 65+ required) --- ## Closed as fixed - [Block FLoC checks in Chrome/uBO](https://github.com/uBlockOrigin/uBlock-issues/issues/1553) - [json-prune doesn't remove all properties by full wildcard](https://github.com/uBlockOrigin/uBlock-issues/issues/1545) ## Notable commits without en entry in the issue tracker - [Remove advanced setting `ignoreScriptInjectFilters`](https://github.com/gorhill/uBlock/commit/bc8c1d00ad728e854b3caf35231d808515058d3f) - [Finalize 3rd-party scripts/frames mini-filtering widget](https://github.com/gorhill/uBlock/commit/58505cfddcabf216fdfdd1150eedfc2f28f36435) - [Remove advanced setting `ignoreRedirectFilters`](https://github.com/gorhill/uBlock/commit/f5b453fae3c77363367cdf9494e90fe276042a72) - [Fix regex used to extract attribute name in element picker](https://github.com/gorhill/uBlock/commit/9a94ba0a226f8d9589806e07409ee6de4925a721) - [Add support for removal of response headers](https://github.com/gorhill/uBlock/commit/f876b68171ff307f27601225607a6801f400437d) 2021-04-16T13:03:59+00:00 netbox v2.11.0 netbox v2.11.0 2021-04-16T13:53:55+00:00 **Note:** NetBox v2.11 is the last major release that will support Python 3.6. Beginning with NetBox v2.12, Python 3.7 or later will be required. ## Breaking Changes * All objects now use numeric IDs in their UI view URLs instead of slugs. You may need to update external references to NetBox objects. (Note that this does _not_ affect the REST API.) * The UI now uses numeric IDs when filtering object lists. You may need to update external links to filtered object lists. (Note that the slug- and name-based filters will continue to work, however the filter selection fields within the UI will not be automatically populated.) * The RackGroup model has been renamed to Location (see [#4971](https://github.com/netbox-community/netbox/issues/4971)). Its REST API endpoint has changed from `/api/dcim/rack-groups/` to `/api/dcim/locations/`. * The foreign key field `group` on dcim.Rack has been renamed to `location`. * The foreign key field `site` on ipam.VLANGroup has been replaced with the `scope` generic foreign key (see [#5284](https://github.com/netbox-community/netbox/issues/5284)). * Custom script ObjectVars no longer support the `queryset` parameter: Use `model` instead (see [#5995](https://github.com/netbox-community/netbox/issues/5995)). ## New Features ### Journaling Support ([#151](https://github.com/netbox-community/netbox/issues/151)) NetBox now supports journaling for all primary objects. The journal is a collection of human-generated notes and comments about an object maintained for historical context. It supplements NetBox's change log to provide additional information about why changes have been made or to convey events which occur outside NetBox. Unlike the change log, in which records typically expire after some time, journal entries persist for the life of the associated object. ### Parent Interface Assignments ([#1519](https://github.com/netbox-community/netbox/issues/1519)) Virtual device and VM interfaces can now be assigned to a "parent" interface by setting the `parent` field on the interface object. This is helpful for associating subinterfaces with their physical counterpart. For example, you might assign virtual interfaces Gi0/0.100 and Gi0/0.200 as children of the physical interface Gi0/0. ### Pre- and Post-Change Snapshots in Webhooks ([#3451](https://github.com/netbox-community/netbox/issues/3451)) In conjunction with the newly improved change logging functionality ([#5913](https://github.com/netbox-community/netbox/issues/5913)), outgoing webhooks now include both pre- and post-change representations of the modified object. These are available in the rendering context as a dictionary named `snapshots` with keys `prechange` and `postchange`. For example, here are the abridged snapshots resulting from renaming a site and changing its status: ```json "snapshots": { "prechange": { "name": "Site 1", "slug": "site-1", "status": "active", ... }, "postchange": { "name": "Site 2", "slug": "site-2", "status": "planned", ... } } ``` Note: The pre-change snapshot for a newly created will always be null, as will the post-change snapshot for a deleted object. ### Mark as Connected Without a Cable ([#3648](https://github.com/netbox-community/netbox/issues/3648)) Cable termination objects (circuit terminations, power feeds, and most device components) can now be marked as "connected" without actually attaching a cable. This helps simplify the process of modeling an infrastructure boundary where we don't necessarily know or care what is connected to an attachment point, but still need to reflect the termination as being occupied. In addition to the new `mark_connected` boolean field, the REST API representation of these objects now also includes a read-only boolean field named `_occupied`. This conveniently returns true if either a cable is attached or `mark_connected` is true. ### Allow Assigning Devices to Locations ([#4971](https://github.com/netbox-community/netbox/issues/4971)) Devices can now be assigned to locations (formerly known as rack groups) within a site without needing to be assigned to a particular rack. This is handy for assigning devices to rooms or floors within a building where racks are not used. The `location` foreign key field has been added to the Device model to support this. ### Dynamic Object Exports ([#4999](https://github.com/netbox-community/netbox/issues/4999)) When exporting a list of objects in NetBox, users now have the option of selecting the "current view". This will render CSV output matching the current configuration of the table being viewed. For example, if you modify the sites list to display only the site name, tenant, and status, the rendered CSV will include only these columns, and they will appear in the order chosen. The legacy static export behavior has been retained to ensure backward compatibility for dependent integrations. However, users are strongly encouraged to adapt custom export templates where needed as this functionality will be removed in v2.12. ### Variable Scope Support for VLAN Groups ([#5284](https://github.com/netbox-community/netbox/issues/5284)) In previous releases, VLAN groups could be assigned only to a site. To afford more flexibility in conveying the true scope of an L2 domain, a VLAN group can now be assigned to a region, site group (new in v2.11), site, location, or rack. VLANs assigned to a group will be available only to devices and virtual machines which exist within its scope. For example, a VLAN within a group assigned to a location will be available only to devices assigned to that location (or one of its child locations), or to a rack within that location. ### New Site Group Model ([#5892](https://github.com/netbox-community/netbox/issues/5892)) This release introduces the new SiteGroup model, which can be used to organize sites similar to the existing Region model. Whereas regions are intended for geographically arranging sites into countries, states, and so on, the new site group model can be used to organize sites by functional role or other arbitrary classification. Using regions and site groups in conjunction provides two dimensions along which sites can be organized, offering greater flexibility to the user. ### Improved Change Logging ([#5913](https://github.com/netbox-community/netbox/issues/5913)) The ObjectChange model (which is used to record the creation, modification, and deletion of NetBox objects) now explicitly records the pre-change and post-change state of each object, rather than only the post-change state. This was done to present a more clear depiction of each change being made, and to prevent the erroneous association of a previous unlogged change with its successor. ### Provider Network Modeling ([#5986](https://github.com/netbox-community/netbox/issues/5986)) A new provider network model has been introduced to represent the boundary of a network that exists outside the scope of NetBox. Each instance of this model must be assigned to a provider, and circuits can now terminate to either provider networks or to sites. The use of this model will likely be extended by future releases to support overlay and virtual circuit modeling. ## Enhancements * [#4833](https://github.com/netbox-community/netbox/issues/4833) - Allow assigning config contexts by device type * [#5344](https://github.com/netbox-community/netbox/issues/5344) - Add support for custom fields in tables * [#5370](https://github.com/netbox-community/netbox/issues/5370) - Extend custom field support to organizational models * [#5375](https://github.com/netbox-community/netbox/issues/5375) - Add `speed` attribute to console port models * [#5401](https://github.com/netbox-community/netbox/issues/5401) - Extend custom field support to device component models * [#5425](https://github.com/netbox-community/netbox/issues/5425) - Create separate tabs for VMs and devices under the cluster view * [#5451](https://github.com/netbox-community/netbox/issues/5451) - Add support for multiple-selection custom fields * [#5608](https://github.com/netbox-community/netbox/issues/5608) - Add REST API endpoint for custom links * [#5610](https://github.com/netbox-community/netbox/issues/5610) - Add REST API endpoint for webhooks * [#5757](https://github.com/netbox-community/netbox/issues/5757) - Add unique identifier to every object view * [#5830](https://github.com/netbox-community/netbox/issues/5830) - Add `as_attachment` to ExportTemplate to control download behavior * [#5848](https://github.com/netbox-community/netbox/issues/5848) - Filter custom fields by content type in format `<app_label>.<model>` * [#5891](https://github.com/netbox-community/netbox/issues/5891) - Add `display` field to all REST API serializers * [#5894](https://github.com/netbox-community/netbox/issues/5894) - Use primary keys when filtering object lists by related objects in the UI * [#5895](https://github.com/netbox-community/netbox/issues/5895) - Rename RackGroup to Location * [#5901](https://github.com/netbox-community/netbox/issues/5901) - Add `created` and `last_updated` fields to device component models * [#5971](https://github.com/netbox-community/netbox/issues/5971) - Add dedicated views for organizational models * [#5972](https://github.com/netbox-community/netbox/issues/5972) - Enable bulk editing for organizational models * [#5975](https://github.com/netbox-community/netbox/issues/5975) - Allow partial (decimal) vCPU allocations for virtual machines * [#6001](https://github.com/netbox-community/netbox/issues/6001) - Paginate component tables under device views * [#6038](https://github.com/netbox-community/netbox/issues/6038) - Include tagged objects list on tag view * [#6088](https://github.com/netbox-community/netbox/issues/6088) - Improved table configuration form * [#6097](https://github.com/netbox-community/netbox/issues/6097) - Redirect old slug-based object views * [#6125](https://github.com/netbox-community/netbox/issues/6125) - Add locations count to home page * [#6146](https://github.com/netbox-community/netbox/issues/6146) - Add bulk disconnect support for power feeds * [#6149](https://github.com/netbox-community/netbox/issues/6149) - Support image attachments for locations ## Bug Fixes (from v2.11-beta1) * [#5583](https://github.com/netbox-community/netbox/issues/5583) - Eliminate redundant change records when adding/removing tags * [#6100](https://github.com/netbox-community/netbox/issues/6100) - Fix VM interfaces table "add interfaces" link * [#6104](https://github.com/netbox-community/netbox/issues/6104) - Fix location column on racks table * [#6105](https://github.com/netbox-community/netbox/issues/6105) - Hide checkboxes for VMs under cluster VMs view * [#6106](https://github.com/netbox-community/netbox/issues/6106) - Allow assigning a virtual interface as the parent of an existing interface * [#6107](https://github.com/netbox-community/netbox/issues/6107) - Fix rack selection field on device form * [#6110](https://github.com/netbox-community/netbox/issues/6110) - Fix handling of TemplateColumn values for table export * [#6123](https://github.com/netbox-community/netbox/issues/6123) - Prevent device from being assigned to mismatched site and location * [#6124](https://github.com/netbox-community/netbox/issues/6124) - Location `parent` filter should return all child locations (not just those directly assigned) * [#6130](https://github.com/netbox-community/netbox/issues/6130) - Improve display of assigned models in custom fields list * [#6155](https://github.com/netbox-community/netbox/issues/6155) - Fix admin links for plugins, background tasks * [#6171](https://github.com/netbox-community/netbox/issues/6171) - Fix display of horizontally-scrolling object lists * [#6173](https://github.com/netbox-community/netbox/issues/6173) - Fix assigned device/VM count when bulk editing/deleting device roles * [#6176](https://github.com/netbox-community/netbox/issues/6176) - Correct position of MAC address field when creating VM interfaces * [#6177](https://github.com/netbox-community/netbox/issues/6177) - Prevent VM interface from being assigned as its own parent ## Other Changes * [#1638](https://github.com/netbox-community/netbox/issues/1638) - Migrate all primary keys to 64-bit integers * [#5873](https://github.com/netbox-community/netbox/issues/5873) - Use numeric IDs in all object URLs * [#5938](https://github.com/netbox-community/netbox/issues/5938) - Deprecated support for Python 3.6 * [#5990](https://github.com/netbox-community/netbox/issues/5990) - Deprecated `display_field` parameter for custom script ObjectVar and MultiObjectVar fields * [#5995](https://github.com/netbox-community/netbox/issues/5995) - Dropped backward compatibility for `queryset` parameter on ObjectVar and MultiObjectVar (use `model` instead) * [#6014](https://github.com/netbox-community/netbox/issues/6014) - Moved the virtual machine interfaces list to a separate view * [#6071](https://github.com/netbox-community/netbox/issues/6071) - Cable traces now traverse circuits ## REST API Changes * All primary keys are now 64-bit integers * All model serializers now include a `display` field to be used for the presentation of an object to a human user * All device components * Added support for custom fields * Added `created` and `last_updated` fields to track object creation and modification * All device component templates * Added `created` and `last_updated` fields to track object creation and modification * All organizational models * Added support for custom fields * All cable termination models (cabled device components, power feeds, and circuit terminations) * Added `mark_connected` boolean field to force connection status * Added `_occupied` read-only boolean field as common attribute for determining whether an object is occupied * Renamed RackGroup to Location * The `/dcim/rack-groups/` endpoint is now `/dcim/locations/` * circuits.CircuitTermination * Added the `provider_network` field * Removed the `connected_endpoint`, `connected_endpoint_type`, and `connected_endpoint_reachable` fields * circuits.ProviderNetwork * Added the `/api/circuits/provider-networks/` endpoint * dcim.Device * Added the `location` field * dcim.Interface * Added the `parent` field * dcim.PowerPanel * Renamed `rack_group` field to `location` * dcim.Rack * Renamed `group` field to `location` * dcim.Site * Added the `group` foreign key field to SiteGroup * dcim.SiteGroup * Added the `/api/dcim/site-groups/` endpoint * extras.ConfigContext * Added the `site_groups` many-to-many field to track the assignment of ConfigContexts to SiteGroups * extras.CustomField * Added new custom field type: `multi-select` * extras.CustomLink * Added the `/api/extras/custom-links/` endpoint * extras.ExportTemplate * Added the `as_attachment` boolean field * extras.ObjectChange * Added the `prechange_data` field * Renamed `object_data` to `postchange_data` * extras.Webhook * Added the `/api/extras/webhooks/` endpoint * ipam.VLANGroup * Added the `scope_type`, `scope_id`, and `scope` fields (`scope` is a generic foreign key) * Dropped the `site` foreign key field * virtualization.VirtualMachine * `vcpus` has been changed from an integer to a decimal value * virtualization.VMInterface * Added the `parent` field 2021-04-16T13:53:55+00:00 faraday v3.14.4 faraday v3.14.4 2021-04-16T17:19:20+00:00 3.14.4 [Apr 15th, 2021]: --- * Updated plugins package, which update appscan plugin 2021-04-16T17:19:20+00:00 cfssl 1.2.0 cfssl 1.2.0 2021-04-16T19:18:27+00:00 2021-04-16T19:18:27+00:00 cfssl 1.1.0 cfssl 1.1.0 2021-04-16T19:20:30+00:00 2021-04-16T19:20:30+00:00 clair v4.0.5 clair v4.0.5 2021-04-16T20:38:47+00:00 <a name="unreleased"></a> ## [Unreleased] <a name="v4.0.5"></a> ## [v4.0.5] - 2021-04-16 ### Chore - [b92ba98](https://github.com/quay/clair/commit/b92ba981540bf13344f5fe48d5683fd2c600e92b): v4.0.5 changelog bump - [486ccfb](https://github.com/quay/clair/commit/486ccfb9d8baac5f468acf0cc0752d7d2d9f8ce4): bump cc stable to v0.1.25 [Unreleased]: https://github.com/quay/clair/compare/v4.0.5...HEAD [v4.0.5]: https://github.com/quay/clair/compare/v4.0.4...v4.0.5 2021-04-16T20:38:47+00:00 PGPy v0.5.4 PGPy v0.5.4 2021-04-16T23:42:17+00:00 Note ---- PGPy v0.5.x is still compatible with Python 2.7 and 3.4. Support for those versions will be dropped in PGPy v0.6.0. Bugs Fixed ---------- * Fixed compatibility break with Python < 3.8 (#368) * Fixed importing ABCs from `collections` (#328) Other Changes ------------- * Documentation updates 2021-04-16T23:42:17+00:00