Advertisement

Advertisement

Web Scraping with Go

Web scraping (Wikipedia entry) is a handy tool to have in your arsenal. It can be useful in a variety of situations, like when a website does not provide an API, or you need to parse and extract web content programmatically. This tutorial walks through using the standard library to perform a variety of tasks like making requests, changing headers, setting cookies, using regular expressions, and parsing URLs. It also covers the basics of the goquery package (a jQuery like tool) to scrape information from an HTML web page on the internet.

If you need to reverse engineering a web application based on the network traffic, it may also be helpful to learn how to do packet capture, injection, and analysis with Gopacket.

If you are downloading and storing content from a site you scrape, you may be interested in working with files in Go.

Taking Command Line Arguments in Java

Taking command line arguments is one of the first things you should learn how to do with a new language. In this tutorial we'll walk through a simple Java program that takes command line arguments. We'll look at how to check if any arguments were passed, access them directly by numerical index, and iterate through each argument provided.

Security with Go - My book now published!

Check out Security with Go, a book I recently wrote, available from Packt Publishing. It covers secure development, red team and blue team topics and is useful for developers and infosec professionals like analysts, investigators, engineers, and pentesters. It's a great book if you want to get to know Go better or if you want to start using Go for security.

Creating Systemd Service Files

systemd is used in many mainstream Linux distributions like Arch Linux, CentOS, Debian/Ubuntu, RedHat/Fedora, openSuse, Slackware, CoreOS and more. It provides an easy way to manage and control services and a simple method of creating your own services. This will cover the process of creating and managing your own custom service.

You should copy your .service file to /etc/systemd/system. Do not symlink it. One, you can't run systemctl enable because you it will not follow a symlink. Two, it potentially opens up a security risk (e.g. a shell). For example, you run your service as a low privilege user but you are symlinking the .service file. Someone finds a flaw in your service where they are able to overwrite or modify files. They can turn that in to code execution by modifying the .service file that your low privilege user has access to and changing the command that is run (ExecStart). When the service is restarted the attackers command is run. This is also why you should not run the service as root.

Note that you can also put the files in /usr/lib/systemd/system/ but that should be reserved for system level packages. Anything in /etc/systemd/system will override it and that is where user changes should go.

Advertisement

Advertisement