We're investigating Kubernetes network overlays at work and I am spinning up sample environments to try things out. One that stands out so far is Cilium due to the fine-grained access controls that can be enforced. They have instructions for how to deploy on Minikube, but it took some finangling for me to be successful with my deployment configuration (Ubuntu 18.04 Server running Minikube 'local' without vagrant).
To cut to the chase, skip to the end to see a deploy script that deploys everything in order.
I've had an idea kicking around in the back of my mind for the last few months to create a Thunderbird extension that will indicate if an email sender's domain was recently registered and alert me. With the poor state of Thunderbird add-on documentation it is a real struggle to get started with anything beyond the most basic 'hello-world' extension. This time I decided to double-down and fight my way through to develop a working (Alpha quality) plugin that accomplishes my design.
If you are thinking about developing an extension for Thunderbird 60 and would like some pointers, read on for my choppy journey through Thunderbird extension development. Hopefully one or more of the pointers will save you time
While setting up a new private docker image registry with certificates signed by an internal certificate authority this week we ran into an issue getting our docker nodes to communicate:
Error response from daemon: Get https://private.registry.tld/v2/: x509: certificate signed by unknown authority
Following the guidance on self-signed certificates from Docker did not directly address the issue.
I was asked to help troubleshoot a NodeJS project recently where the team was encountering trouble connecting to an elasticsearch instance securely (via https/tls). They would get an error back about 'self signed certificate in certificate chain`. In examining further, we were able to come up with a client configuration for the elasticsearch library that addressed the issue.
Yesterday marked a first for me: I had to restore a few objects from a large S3 bucket that was backed up to Glacier. Along the way I learned a few things:
- Objects sent to glacier permanently retain the GLACIER storage class
- If your S3 objects were replicated across an AWS Account boundary, you might not have 'full control' of your objects (but AWS will gladly let you pay them to store them)
- The AWS CLI is unhelpful when it comes to recursively copying objects that are restored from glacier
The objects can be restored and downloaded, it just takes some specific knowledge
This past week I've been getting to know the Qualys API by writing an integration with Go. Along the way I've found some quirks that are worth mentioning for anyone getting to know the Qualys platform at the API level.
To jump straight to the sample code repository, you can go to the qualys-api-samples repo on GitHub.
Over the last year I've been learning the Go programming language and overall it's been a pretty positive experience- except when it comes to quickly and easily interfacing with databases. While Go does include a 'sql' package targeted at low level interaction with database backends, out of the box you are not provided with something higher level (think rails ActiveRecord). For higher level abstraction there are many community supported packages available, which makes it time consuming to try/test each one and see if it fits your needs. Today I'm writing about SQLBoiler, specifically about working with its SQLite integration.
Update (2018-10-25): Now includes instructions for building sqlboiler as well as sqlboiler-sqlite3 and ensuring both of them are in the same directory or system path before usage.
While performing a security assessment this last week I found that the applications ran in a containerized environment on AWS EC2 instances. The EC2 meta-data service was available to be queried from within these application containers (not a best practice) and exposes a rich array of information for any would-be attackers who can gain a shell into the container or cause the application to perform SSRF against the local metadata service endpoint.
I wanted a quick way to query all of the meta data and user-data exposed by the EC2 meta-data service and created this metadata extractor script.
I'm a big fan of the Small World board game and was pleased to see they have a version available on Steam. Unfortunately it did not run correctly on my primary system (Ubuntu 16.04). In fact, it did not start at all. When I ran it at the CLI I found error messages that aided me in my quest to get the game running. Read on for the details.
This is my 6th DEF CON and I plan on coming back for more! There is a lot of life and energy at the con that I haven't been able to find at other conferences. A big appeal to me is that DEF CON itself is kind of a wrapper event where you find a number of mini-conferences (called Villages), so even if the main tracks don't interest you, odds are you'll find something at the 27-ish villages that run at the same time.
This year's Black Hat USA conference was pretty solid. Every timeslot had something available that I found interesting and often times I had to pick between competing sessions that captured my interest. Conference organization and crowd control was excellent again as usual. The mobile app this year had more features than in previous years, which I appreciate, though it does lose a few points for complicated and missing capabilities.
I had the chance to attend Black Hat this year and attend a 2 day of my choice. This year I took the Advanced Infrastructure Hacking - 2018 Edition: 2 Day session sponsored by NOTSOSECURE. It can be hard to find reviews of these trainings so I think it's worthwhile to post my thoughts here.
My notes from RSA 2018 sessions and labs. I've sanded off the rough-edges from my raw notes. Might still be a bit 'bumpy'
When the opportunity to attend RSA came up, I decided to take a chance to see how it compares to other security conferences I've attended in the past (like DEF CON, BlackHat and BSides). After attending the 5 days of the conference I can say that RSA definitely has a different 'feel' to it than the other conferences and depending on what you are trying to accomplish, it may or may not be a good value for the $$$ spent.
We ran into an interesting situation on a legacy system where we were unable to allow outbound traffic on a CentOS 6 server to the internet, yet we needed to install Python 3.4 and the 'requests' library on the server.
OpenVAS is an open source vulnerability scanner that I have used (and seen used) over the last few years. It's history goes back to 2005 as a fork of a previously open source (now commercialized) vulnerability scanner. This tool tends to be used when the dollar-cost of a commercial solution appears to outweigh the time and effort needed to maintain an effective OpenVAS installation.
The most common problem that I encounter using OpenVAS is the 503: service temporarily down error. When I see this message it almost invariably ties back to an expired self-signed certificate. I've seen this error enough times that I want to document the process in case I end up using this tool again in the future.
Awhile ago at a previous employer I worked with a VAR to do a system and software inventory of our workstations. They had written a custom application in C#.NET (for windows systems) and a bash script (for Mac OS) that captured the inventory data and encrypted it for us to email back to them. Sounds pretty straight forward, right?
I asked them about how the data would be protected on collection and being transmitted to them and surprisingly heard back from the VAR that their encryption mechanism 'cannot be disclosed publicly'. Hmm.... Before agreeing to run the programs and send back results, I performed a secure code review and found some interesting things about their collection tools.
This morning I tried to run security updates on one of my Centos VPS systems. Had to get creative since just running yum upgrade did not work. The yum process was killed unexpectedly:
Transaction Summary ========================== Upgrade 19 Package(s) Total size: 24 M Is this ok [y/N]: y Downloading Packages: Running rpm_check_debug Killed
A month or so ago I started an SSH Statistics gatherer with the hope of identifying high-level configuration details of SSH-2/SSH-1.99 servers in the USA. In running the tool for a couple of weeks I identified 46,250 SSH Servers that meet the basic criteria (I'd like to do a survey of SSH Servers running older versions in the 1.x range at a later date). This post explains the results of the survey.
Today I spent way too much time trying to debug an issue encountered while developing a Chrome extension. While attempting to create an Options page, I setup a separate 'options.js' file (to comply with security requirements that don't permit inline-JS) and found that the .js file would not load and that there were no error messages listed in the chrome developer tools view.
An interesting problem surfaced earlier this year that prevented our systems administrators from using RDP to connect to a windows server if they use a windows laptop. Paradoxically, SysAdmins who run Macs were not affected. It took a little time to track this down and now that I've been through the troubleshooting process I know how to fix it and can see how we wound up in this situation in the first place
We switched from LastPass to 1Password and encountered an unexpected hindrance: HTML encoded strings somehow replaced certain characters in critical passwords. The first time through the process it seemed like 1Password was was causing the problem. Upon further investigation we found that the problem originated during the LastPass 'export' process.
I will be starting an SSH Statistics gatherer that will be targeting US based IP addresses today. The gatherer tool will run for 1 week through Sunday, January 22, 2017. During this time you may notice SSH-2.0-ssh-stats-gather-2017_1.0.0 appear in your SSH server logs. This tool performs a banner grab of SSH servers and does not attempt to login (performs a partial connect)
I will update this post once the run completes with more details.
Update 2017-01-29: Things picked up pretty fast and I was able to pick up quite a bit of data. The stats gathering tool has been turned off and I am parsing the results. Expect a post about the details at some point in the next few weeks.
Update 2017-01-22: Technical issues have come up which require that I extend the duration of this project for another couple of weeks. The new target completion date is Sunday, February 5th, 2017
Update 2017-02-21: Results were published here
This week I needed to compile PuTTY to work on an Ubuntu system running 16.04 (LTS). The instructions are pretty straight-forward and will take you most of the way through compiling something you can use. A problem I ran into is that I kept on running into errors during compilation referencing dlsym, dlopen and dlclose.
Fate was on my side as I was able to work my way through the problem (with a generous dose of google) and comple that actually works.
For the last few months we have been experiencing intermittent issues with one of our production processes. The issue is one that has confounded us in its lack of consistency and ability to frustrate anyone assigned to troubleshoot the problem. I was asked to look into the situation and in the end was able to discover the root cause in just a few hours using Telerik JustDecompile coupled with Microsoft's SQL Profiler tool. The journey was exciting and I'll share what I can here.