Logging runs for a while, then mysteriously stops

Murus
Post Reply
jporten
Posts: 9
Joined: Fri Nov 03, 2017 3:46 pm

Logging runs for a while, then mysteriously stops

Post by jporten » Mon Nov 27, 2017 1:22 am

We are running Murus with no firewalling for the time being, with the idea of collecting insanely detailed logs so we know what to firewall later (at which time, we'll throttle the logs to something sane).

The problem: after a few hours or a few days, all logging stops. Can't find a reason why, and in fact, this happened over Thanksgiving when no human even looked at the server. We are running 10.12.6, so I do not know if the patch published here for 10.12.4 is a good idea.

Help?

jporten
Posts: 9
Joined: Fri Nov 03, 2017 3:46 pm

Re: Logging runs for a while, then mysteriously stops

Post by jporten » Mon Nov 27, 2017 1:38 am

Documenting some additional log weirdness. We had the maximum log settings before we left for Thanksgiving; since then, I've applied hany's edit to pf.conf to increase the number of saved logs to something like 200, but since we're not in the office we haven't rebooted the server yet, and I didn't expect to see them take hold. I have an AppleScript application in the meantime to poll /var/log for new log files ending in .bz2, and we're copying those to a Desktop folder until we can reboot.

Checking /var/log, before the logging stopped the first time on Nov 7, it actually went up to log.29 at one point, and that file was never zipped. But we have more recent files up to log.20, so apparently it rotated up to 29 (20 is in the UI settings) at some point, and then stopped.

In short, I have absolutely no idea what's going on with pf logging. I've stopped and started pf, and it's not logging anything—we have *everything* set to log, inbound and outbound.

hany
Posts: 457
Joined: Wed Dec 10, 2014 5:20 pm

Re: Logging runs for a while, then mysteriously stops

Post by hany » Tue Nov 28, 2017 9:02 pm

PF logging on macOS before 10.12 used tcpdump to read and store logs.
Then when macOS 10.12 came out we we were forced to switch to a new logging system because tcpdump on 10.12 is bugged and it does not work.
So Murus on 10.12 uses pfloggerd instead of tcpdump. That may be the reason, probably there is a bug on pfloggerd. We never identified it before.
The weird thing is that it seems that tcpdump is working now on macOS 10.13, so we will probably go back to tcpdump in next Murus versions. We are still unsure about what to do with 10.12.
Please let us make some tests with pfloggerd. But as you can imagine it's not easy to reproduce a bug that occurs "every now and then" :)

jporten
Posts: 9
Joined: Fri Nov 03, 2017 3:46 pm

Re: Logging runs for a while, then mysteriously stops

Post by jporten » Mon Jul 02, 2018 1:27 am

Just checking in to see if there's any progress re this problem with 10.12? We've been on hold for Murus implementation for a while, but now kicking into gear. We'll almost certainly skip 10.13 entirely, and *maybe* go to 10.14.1 depending on how Server changes impact us—otherwise we'll be staying here for a while.

jewettg
Posts: 7
Joined: Wed Sep 26, 2018 4:02 pm

Re: Logging runs for a while, then mysteriously stops

Post by jewettg » Mon Oct 08, 2018 1:49 pm

@jporten I am at version 10.13.6, and the logging continues to be an issue. I have a post open and @hany has responded, but I am still stumped. I can not keep the process running without it crashing. Still looking for help.

jewettg
Posts: 7
Joined: Wed Sep 26, 2018 4:02 pm

Re: Logging runs for a while, then mysteriously stops

Post by jewettg » Mon Oct 08, 2018 1:59 pm

jewettg wrote: @jporten I am at version 10.13.6, and the logging continues to be an issue. I have a post open and @hany has responded, but I am still stumped. I can not keep the process running without it crashing. Still looking for help.
Hang on - 1.4.19 of Murus has been released, checking to see if this fixes bugs.
Changes: Runs on macOS 10.14 Mojave; Minor bugs fixed

hany
Posts: 457
Joined: Wed Dec 10, 2014 5:20 pm

Re: Logging runs for a while, then mysteriously stops

Post by hany » Tue Oct 09, 2018 1:39 pm

We are working at Murus 2 which uses a totally different logging system, based on a mysql database. It currently seams more reliable and powerful than the old system. You don't need to wait for Murus 2 to test this system, we will release a package that will enable it on Murus 1 as well. If you are interested in the closed beta testing just write us at info@murus.it, if you want to wait for the public beta then please stay tuned!

jewettg
Posts: 7
Joined: Wed Sep 26, 2018 4:02 pm

Re: Logging runs for a while, then mysteriously stops

Post by jewettg » Tue Oct 09, 2018 1:53 pm

OK! Thank you @hany, the logging daemon seems stable, and logging. It was odd last night.. I got dinged, b/c the FW was turned off. I would have never done that, which puzzles me. I will keep watching it.

Post Reply