Development Notes

Toolchains - DevOps tools for various languages

LanguageBuilderPackage managerVersion managerTester
Python pipvirtualenv 
Ruby gemrvm 

Static file extensions

location ~* \.(7z|ai|bmp|bz2|class|css|csv|docx?|ejs|eot|eps|flv|gif|gz|html?|ico|jar|jpe?g|js|json|lzh|m4a|m4v|midi?|mov|mp3|mp4|pdf|pict|pls|png|pptx?|ps|psd|rar|rss|rtf|svgz?|swf|tar|tiff?|ttf|txt|wav|webp|woff|xlsx?|zip)$

Locks and Semaphores

Locks are analogous to green traffic lights: If you have a green light, that does not prevent the idiot coming the other way from plowing into you sideways; it merely guarantees to you that the idiot does not also have a green light at the same time.


Optimistic UI

So why do we want to define our methods on the client and on the server? We do this to enable a feature we call optimistic UI.

When you call a method on the client using, two things happen in parallel:

The client sends a request to the server to run the method in a secure environment, just like an AJAX request would work

What this means is that a newly created task actually appears on the screen before the result comes back from the server.

If the result from the server comes back and is consistent with the simulation on the client, everything remains as is. If the result on the server is different from the result of the simulation on the client, the UI is patched to reflect the actual state of the server.

Templating systems

Blaze. It's a templating system similar to, but simpler than Angular, Backbone, Ember, React (uses JSX by facebook), Polymer, and Knockout.

Blaze lacks when it comes to handling local/internal state while React has this nailed down very, very well with props and composing nested component hierarchies, with data flowing from higher components to lower components
This translates into a much higher learning curve for React, but definite benefits down the road in terms of your codebase’s efficiency and maintainability.

Liquid templating - used by jekyll, created by shopify

Python binary data

The Bytes Type

The bytes type in Python is immutable and stores a sequence of values ranging from 0-255 (8-bits). You can get the value of a single byte by using an index like an array, but the values can not be modified.

# Create empty bytes
empty_bytes = bytes(4)

The Bytearray Type

To create a mutable object you need to use the bytearray type. With a bytearray you can do everything you can with other mutables like push, pop, insert, append, delete, and sort.

# Cast bytes to bytearray
mutable_bytes = bytearray(b'\x00\x0F')

# Bytearray allows modification
mutable_bytes[0] = 255

# Cast bytearray back to bytes
immutable_bytes = bytes(mutable_bytes)

Reading Bytes From a File

with open("test_file.dat", "rb") as binary_file:

# Read the whole file at once
data =
# Seek position and read N bytes # Go to beginning
couple_bytes =

Integer to Bytes

i = 16

# Create one byte from the integer 16
single_byte = i.to_bytes(1, byteorder='big', signed=True)

# Create four bytes from the integer
four_bytes = i.to_bytes(4, byteorder='big', signed=True)

# Compare the difference to little endian
print(i.to_bytes(4, byteorder='little', signed=True))

# Create bytes from a list of integers with values from 0-255
bytes_from_list = bytes([255, 254, 253, 252])

# Create a byte from a base 2 integer
one_byte = int('11110000', 2)

# Print out binary string (e.g. 0b010010)

Bytes to Integer

# Create an int from bytes. Default is unsigned.
some_bytes = b'\x00\xF0'
i = int.from_bytes(some_bytes, byteorder='big')

# Create a signed int
i = int.from_bytes(b'\x00\x0F', byteorder='big', signed=True)

# Use a list of integers 0-255 as a source of byte values
i = int.from_bytes([255, 0, 0, 0], byteorder='big')

Text Encoding

# Binary to Text
binary_data = b'I am text.'
text = binary_data.decode('utf-8')

binary_data = bytes([65, 66, 67]) # ASCII values for A, B, C
text = binary_data.decode('utf-8')

# Text to Binary
message = "Hello" # str
binary_message = message.encode('utf-8')
print(type(binary_message)) # bytes

# Python has many built in encodings for different languages,
# and even the Caeser cipher is built in
import codecs
cipher_text = codecs.encode(message, 'rot_13')

Base 64 Encoding

# Encode binary data to a base 64 string
binary_data = b'\x00\xFF\x00\xFF'

# Use the codecs module to encode
import codecs
base64_data = codecs.encode(binary_data, 'base64')

# Or use the binascii module
import binascii
base64_data = binascii.b2a_base64(binary_data)

# The base64_string is still a bytes type
# It may need to be decoded to an ASCII string

# Decoding is done similarly
print(codecs.decode(base64_data, 'base64'))


# Starting with a hex string you can unhexlify it to bytes
deadbeef = binascii.unhexlify('DEADBEEF')

# Given raw bytes, get an ASCII string representing the hex values
hex_data = binascii.hexlify(b'\x00\xff') # Two bytes values 0 and 255

# The resulting value will be an ASCII string but it will be a bytes type
# It may be necessary to decode it to a regular string
text_string = hex_data.decode('utf-8') # Result is string "00ff"

Format Strings

Format strings can be helpful to visualize or output byte values. Format strings require an integer value so the byte will have to be converted to an integer first.

a_byte = b'\xff' # 255
i = ord(a_byte) # Get the integer value of the byte

bin = "{0:b}".format(i) # binary: 11111111
hex = "{0:x}".format(i) # hexadecimal: ff
oct = "{0:o}".format(i) # octal: 377


Bitwise Operations

# Some bytes to play with
byte1 = int('11110000', 2) # 240
byte2 = int('00001111', 2) # 15
byte3 = int('01010101', 2) # 85

# Ones Complement (Flip the bits)

print(byte1 & byte2)

# OR
print(byte1 | byte2)

print(byte1 ^ byte3)

# Shifting right will lose the right-most bit
print(byte2 >> 3)

# Shifting left will add a 0 bit on the right side
print(byte2 << 1)

# See if a single bit is set
bit_mask = int('00000001', 2) # Bit 1
print(bit_mask & byte1) # Is bit set in byte1?
print(bit_mask & byte2) # Is bit set in byte2?

Struct Packing and Unpacking

Packing and unpacking requires a string that defines how the binary data is structured. It needs to know which bytes represent values. It needs to know whether the entire set of bytes represets characters or if it is a sequence of 4-byte integers. It can be structured in any number of ways. The format strings can be simple or complex. In this example I am packing a single four-byte integer followed by two characters. The letters i and c represent integers and characters.

import struct

# Packing values to bytes
# The first parameter is the format string. Here it specifies the data is structured
# with a single four-byte integer followed by two characters.
# The rest of the parameters are the values for each item in order
binary_data = struct.pack("icc", 8499000, b'A', b'Z')

# When unpacking, you receive a tuple of all data in the same order
tuple_of_data = struct.unpack("icc", binary_data)

# For more information on format strings and endiannes, refer to
# - Does the file have a JPEG binary signature?

import sys
import binascii

jpeg_signatures = [


with open(sys.argv[1], 'rb') as file:

if first_four_bytes in jpeg_signatures:
print("JPEG detected.")

# - Identify ASCII characters in binary files

import sys
from functools import partial

chunk_size = 1
with open(sys.argv[1], 'rb') as in_file:

for data in iter(partial(, chunk_size), b''):
x = int.from_bytes(data, byteorder='big')
if (x > 64 and x < 91) or (x > 96 and x < 123) :

# Extract PNGs from a file and put them in a pngs/ directory
import sys
with open(sys.argv[1], "rb") as binary_file:, 2) # Seek the end
count = 0
for i in range(num_bytes):
eight_bytes =
if eight_bytes == b"\x89\x50\x4e\x47\x0d\x0a\x1a\x0a": # PNG signature
count += 1
# Next four bytes after signature is the IHDR with the length
png_size_bytes =
# Go back to beginning of image file and extract full thing
# Read the size of image plus the signature
png_data = + 8)
with open("pngs/" + str(i) + ".png", "wb") as outfile:

Android SDK

get from

vi /etc/udev/rules.d/51-android.rules
SUBSYSTEM=="usb", SYSFS{idVendor}=="18d1", MODE="0666"

unzip and run tools/android
pick the targets and install

HTML Site themes

Generic themes (can be converted to Jekyll)

Perl printenv code

print "Content-type: text/html\n\n";
while (($key, $val) = each %ENV) {
        $environ{$key} = $val;
foreach $key (sort keys (%environ)) {
        print "$key = $environ{$key}<BR>\n";
if ($environ{"REQUEST_METHOD"} eq "POST") {
   print "Input from STDIN:<P>";
   print "$line<BR>" while ($line = <STDIN>);

Tomcat component architecture

Catalina is actually the servlet container, Coyote is the web-server-esque connector which forwards the requests onto Catalina. Web pages are then delivered dynamically through Jasper, the JSP engine. So, in summary Tomcat is the sum of Catalina, Coyote, and Jasper.

So, Tomcat itself can serve as a web server. Its Coyote component handles the requests, forwards them onto Catalina, which then allows Jasper to serve up a JSP.

@TechnicalNotes @Development