profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/wagner-certat/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Wagner wagner-certat @certat Vienna https://cert.at/ Also known as @sebix

CZ-NIC/python-rt 44

Python interface to Request Tracker API

enisaeu/NIS-sectors 3

A public list of Sectors as defined by the NIS-Directive

certat/certspotter-processing 2

A bunch of short scripts used for handing the results of the program certspotter.

wagner-certat/intelmq-manager 1

IntelMQ Manager is a graphical interface to manage configurations for IntelMQ framework.

wagner-certat/mmquery 1

A command line utility for querying the MatterMost API for various auditing or reporting purposes

certat/npm-audit-checkmk 0

NPM Audit Interpreter and Check MK output generator

sebix/guake 0

Drop-down terminal for GNOME

wagner-certat/awesome-csirt 0

Awesome CSIRT is an curated list of links and resources in security and CSIRT daily activities.

wagner-certat/cerebrate 0

The Cerebrate Sync Platform core software

push eventcerttools/intelmq-tutorial

Sebastian Wagner

commit sha af591f536df5053fbafc66c64acf22609c8e18b9

packer build spec: specify checksum type otherwise packer fails with * The iso_checksum_type must be specified.

view details

push time in 19 days

push eventcerttools/intelmq

Sebastian Wagner

commit sha 5962fa9efb1c9d1ae454cc85d4aa05ddfe3d1080

PKG: setup.py: Change maintainer to intelmq-dev list

view details

push time in 19 days

startedhashlookup/hashlookup-lib

started time in 21 days

issue commentcerttools/intelmq

Connecting Intelmq Manager with User database (IntelMQv3)

Closing because of inactivity. Please reopen if needed.

YeomansM

comment created time in 24 days

issue closedcerttools/intelmq

Connecting Intelmq Manager with User database (IntelMQv3)

Good Afternoon Guys,

CSIRTMalta is trying to install on a blank system (Ubuntu Server 20.04) the new version of IntelMQ v3. Downloaded and installed only through repository packages. ( Preamble Intelmqctl worked only after rwamel.yaml was installed through pip3.)

Summary of the problem is that the intelmq manager cannot call and/or connect with the SQLite DB.

  • We followed the notes for the Path for Session store within api-config.json which had to be modified for Ubuntu 20.04. WIthin the default configuration file ( api-config.json) the provided Path was /var/lib/dbconfig-common/sqlite3/intelmq-api/intelmqapi. Were we correct that from our end we modified it to the follow path /etc/intelmq/api-session.sqlite?

  • Furthermore, we have checked the logs within access.log and found the follow line hereunder. IP Address - - [10/Aug/2021:10:09:55 +0000] "GET /in-telmq/v1/api/login HTTP/1.1" 404 494 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0" Is that a typo which misleads the GET function or it should be read as is correct in-telmq ?

closed time in 24 days

YeomansM

issue commentcerttools/intelmq

ASN Lookup expert database update fails

That fails at the end of every month. The monthly directories in http://archive.routeviews.org/route-views4/bgpdata/ are always created on the 28th of the previous month and thus our code fails on every 29th, 30th and 31st of each month.

wagner-certat

comment created time in 24 days

issue closedcerttools/intelmq

Issues with a clean pip install of intelmq on RHEL 8

Hi

We encounter some problems with installing intelmq using pip on rhel8 where there seems to be a mismatch between the pip installation and source available on github under the 3.0.x tag.

bash-4.4$ pip3 --version pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6) bash-4.4$ pip3 show intelmq Name: intelmq Version: 3.0.2 Summary: IntelMQ is a solution for IT security teams for collecting and processing security feeds using a message queuing protocol. Home-page: https://github.com/certtools/intelmq/ Author: IntelMQ Community Author-email: None License: AGPLv3 Location: /usr/local/lib/python3.6/site-packages Requires: redis, ruamel.yaml, requests, pytz, python-dateutil, dnspython, python-termstyle, psutil bash-4.4$ python3 --version Python 3.6.8

Commands such as "Intelmqctl check" and "intelmqctl list bots" fails in a clean installation when installing the latest release v3.0.2 using pip. It seems that the error stems from expert bots present when using pip install that relies on logic from commit 7d4b9be6bd81d7f3b7750e1aae6be70156dfc3c4 in develop that defines the Class ExpertBot.

grep -r "import ExpertBot" bots/experts/domain_valid/expert.py:from intelmq.lib.bot import ExpertBot bots/experts/truncate_by_delimiter/expert.py:from intelmq.lib.bot import ExpertBot

example output from "intelmqctl check":

intelmqctl check Reading configuration files. Checking runtime and pipeline configuration. Checking harmonization configuration. Checking for bots. Traceback (most recent call last): File "/usr/local/bin/intelmqctl", line 11, in <module> sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/intelmq/bin/intelmqctl.py", line 1909, in main return x.run() File "/usr/local/lib/python3.6/site-packages/intelmq/bin/intelmqctl.py", line 1048, in run retval, results = args.func(**args_dict) File "/usr/local/lib/python3.6/site-packages/intelmq/bin/intelmqctl.py", line 1564, in check for group in utils.list_all_bots().values(): File "/usr/local/lib/python3.6/site-packages/intelmq/lib/utils.py", line 855, in list_all_bots mod = importlib.import_module('.'.join(file.with_suffix('').parts)) File "/usr/lib64/python3.6/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 678, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/usr/local/lib/python3.6/site-packages/intelmq/bots/experts/domain_valid/expert.py", line 20, in <module> from intelmq.lib.bot import ExpertBot ImportError: cannot import name 'ExpertBot'

Furthermore, there seems to be other issues with the release where there is a mismatch between bots available with the 3.0.x release tag in github differs from what is delivered when installing with pip. E.g in github the expert bot ripe is available, with pip install both expert bot "ripe" and "ripencc_abuse_contact" is present.

ls bots/experts/ripe* bots/experts/ripe: expert.py init.py pycache REQUIREMENTS.txt bots/experts/ripencc_abuse_contact: expert.py init.py pycache REQUIREMENTS.txt

Removing the domain_valid and truncate_by_delimiter bots from the install allows "intelmqctl list bots" and "intelmqctl check" commands to complete. However, the check command gives error on the additional bots available in the pip install

Checking for bots. Incomplete installation: Executable 'intelmq.bots.collectors.xmpp.collector' for 'XMPP' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.abusech.parser_ransomware' for 'AbuseCHRansomwaretracker' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.bitcash.parser' for 'BitcashBlocklist' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.fraunhofer.parser_ddosattack_cnc' for 'FraunhoferDdosAttackCnc' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.fraunhofer.parser_ddosattack_target' for 'FraunhoferDdosAttackTarget' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.hphosts.parser' for 'HpHosts' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.malwaredomainlist.parser' for 'MalwareDomainList' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.malwaredomains.parser' for 'MalwareDomains' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.nothink.parser' for 'Nothink' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.parsers.urlvir.parser' for 'URLVir' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.experts.jinja.expert' for 'Jinja' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.experts.ripencc_abuse_contact.expert' for 'RIPENCCExpertDeprecated' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Incomplete installation: Executable 'intelmq.bots.outputs.xmpp.output' for 'XMPP' not found in $PATH ('/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'). Some issues have been found, please check the above output.

best regards, Ole Kristoffer Dybvik Apeland Nkom EkomCERT

closed time in 24 days

olekristoffer

issue commentcerttools/intelmq

Issues with a clean pip install of intelmq on RHEL 8

The release files on PyPI are fixed, so closing here.

olekristoffer

comment created time in 24 days

issue commentcerttools/intelmq

Issues with a clean pip install of intelmq on RHEL 8

Thanks for the feedback. I believe that removing the local build/ folder before generating the release tarballs for PyPI prevents all of the described errors - the missing executables and bots which shouldn't be there - in the future (I've documented that for myself in 2da82cfa5c26cec613858d613f8e96f760b53f1f). I will also remove the bogus wheel package from PyPI tomorrow and replace it with a clean one.

Sorry for the troubles and thanks for the very detailed report and quick and smooth cooperation!

olekristoffer

comment created time in 25 days

issue closedCZ-NIC/python-rt

Support for cookiejar file/variables

Add support for existing cookies, so the login is not necessary.

closed time in 25 days

wagner-certat

issue commentCZ-NIC/python-rt

Support for cookiejar file/variables

Solved by https://github.com/CZ-NIC/python-rt/pull/60

wagner-certat

comment created time in 25 days

issue commentCZ-NIC/python-rt

RT cookie authentification

Do the examples in https://github.com/CZ-NIC/python-rt/pull/60 help?

yshpishak

comment created time in 25 days

issue commentcerttools/intelmq-manager

The PyPi wheels contain outdated code

Maybe the problem was equal to the one which resulted in https://github.com/certtools/intelmq/issues/2116. Apparently python3 setup.py sdist bdist_wheel use the code in the build/ directory and cleaning that one seems to be mandatory.

monoidic

comment created time in 25 days

push eventcerttools/intelmq

Sebastian Wagner

commit sha 2da82cfa5c26cec613858d613f8e96f760b53f1f

doc: release: remove build dir before creating tarballs certtools/intelmq#2116 revealed that not removing the build directory may result in different files being packaged.

view details

push time in 25 days

push eventcerttools/intelmq

Sebastian Wagner

commit sha 64d150d86fad85b00eee8022ec52dd2086902112

DOC: fix duplicate entries in bots documentation

view details

push time in 25 days

issue commentcerttools/intelmq

Issues with a clean pip install of intelmq

I wonder why the two bots (domain_valid and truncate_by_delimiter) are at all in the PyPI package. Removing them is the correct workaround.

It looks like I need to rm -r build/ before creating the packages for PyPI :/ I've re-uploaded the 3.0.2 sdist tarball with name intelmq-3.0.2-post1.tar.gz (I can't use the original name, PyPI prohibits that) without the two bots which shouldn't be there. Can you please re-try?

olekristoffer

comment created time in 25 days

issue commentcerttools/intelmq

Issues with a clean pip install of intelmq

I wonder why the two bots (domain_valid and truncate_by_delimiter) are at all in the PyPI package. Removing them is the correct workaround.

The Incomplete installation issues come from a failed pip installation. pip install intelmq needs to create the executables, that's nothing that intelmq can do itself. But I have no idea, why pip doesn't do that.

olekristoffer

comment created time in a month

delete branch certtools/intelmq

delete branch : wagner/fix-2064

delete time in a month

issue closedcerttools/intelmq

intelmqsetup: Support for intelmq-manager

https://github.com/certtools/intelmq/blob/develop/intelmq/bin/intelmqsetup.py currently supports the IntelMQ Core and the API. Support for the Manager shall be added.

The installation of the Manager is currently tricky and has flaws.

  • On wheel/sdist build, the HTML files are generated, the files also contain the sources
  • pip can't install the html files to the correct location, they land in /usr/local/lib/python3/
  • the destination directory is not fixed, it depends on the distribution (just like with the API files)

This is not good usability. Installation via deb/rpm packages or docker is recommended, but not limited to them. With our own installation method, we can fix the above issues.

There's one disadvantage: When the build happens during installation, the build dependencies need to be installed. As the user used pip anyway, installing mako as dependency is OK.

closed time in a month

wagner-certat

push eventcerttools/intelmq

Sebastian Wagner

commit sha e1ee20215f6274f21de9d25e528571e8586ffc69

ENH: intelmqsetup: revise installation of manager requires certtools/intelmq-manager#282 fixes certtools/intelmq#2064 updates, fixes and re-structures the documentation

view details

push time in a month

PR merged certtools/intelmq

ENH: intelmqsetup: revise installation of manager documentation usability

requires certtools/intelmq-manager#282 updates, fixes and re-structures the documentation

+80 -42

1 comment

4 changed files

wagner-certat

pr closed time in a month

pull request commentcerttools/intelmq-manager

PKG: separate building process from setup.py

Thanks @b1rger !

wagner-certat

comment created time in a month

delete branch certtools/intelmq-manager

delete branch : wagner/build-restructure

delete time in a month

PR merged certtools/intelmq-manager

PKG: separate building process from setup.py packaging

move the building process from setup.py to a separate file, make it an executable ship the apache configuration file if possible adapt the debian build rules

+109 -47

3 comments

9 changed files

wagner-certat

pr closed time in a month

push eventcerttools/intelmq-manager

Sebastian Wagner

commit sha 2f939792bc88a09aac22b629efc61aa769342611

PKG: separate building process from setup.py move the building process from setup.py to a separate file, make it an executable ship the apache configuration file if possible adapt the debian build rules

view details

push time in a month

PR opened certtools/intelmq

Reviewers
DOC: enhance misp integration document documentation

add short intro for explanations on differences link to the bots

+23 -11

0 comment

2 changed files

pr created time in a month

create barnchcerttools/intelmq

branch : wagner/misp-docs

created branch time in a month

pull request commentcerttools/intelmq

Add new bot: cut string from string

Thank you very much - again! - for this useful addition.

mariuskarotkis

comment created time in a month

push eventcerttools/intelmq

Sebastian Wagner

commit sha 1c06ba4e8a75e138797934106b66115133db02b7

DOC: changelog entry for PR#1965

view details

push time in a month

push eventcerttools/intelmq

Marius Karotkis

commit sha 6d7ab0809bcdd2d217cf43db84f8ae45da970a49

Add new bot: cut string from string (#1965) * Add new bot: cut string from string * Add documentation * Change int to bool * change field name, remove init function * Small fix * Update documentation * Upda te bot and documentation * Add license * Fix space * Fix for python 3.6 * Rename bot * Rename bot

view details

push time in a month