I am Joshua Poehls. Say hello Archives (not so) silent thoughts

Languages Are Beautiful

If you ever see me doodling, odds are that I’m drawing letters and numbers of some kind. I’ve always found alphabets, notations, and writing systems in general to be facinating. Typography is beautiful. Typography in foreign languages, even more so.

Jesus Christ, the Son of the Living God

Jesus Christ, the Son of the Living God

Hebrew Alphabet Poster - Red Blocks

Hebrew Alphabet Poster - Red Blocks

Hebrew Alphabet Poster - Classic Blue

Hebrew Alphabet Poster - Classic Blue

See the World by John Yunker

See the World by John Yunker

Hebrew Calligraphy

Hebrew Calligraphy

Writing Systems Poster by Emily Law

Writing Systems Poster by Emily Law

Hebrew Lettering by Tal Maimon - הגיעו מים עד גועל נפש

Hebrew Lettering by Tal Maimon - הגיעו מים עד גועל נפש

⦿

The NSA Reaches Out

Anne Neuberger is Special Assistant to NSA Director Michael Rogers and also the Director of the NSA’s Commercial Solutions Center.

The NSA, Neuberger said, has suffered a particularly “long and challenging year” dealing with the public loss of trust following the Snowden revelations. The agency is reviewing all of its activities to determine how to regain that trust. One change is more open engagement with the public. “This presentation is a starting point.”

Anne has a pretty potent family history and she used it to help build sympathy for her position.

Of her eight great-grandparents, seven were murdered at Auschwitz. “So my family’s history burned into me a fear of what occurs when the power of a state is turned against its people or other people.”

“My family history,” she said, “instilled in me almost parallel value systems – fear of potential for overreach by government, and belief that sometimes only government, with its military and intelligence, can keep civilians safe. Those tensions shape the way I approach my work each day. I fully believe that the two seemingly contradictory factors can be held in balance. And with your help I think we can define a future where they are.”

The seminar was worth listening to if for no other reason than the novelty of hearing a secretive organization talk publicly about itself. Anne certainly spoke well.

“My call to action for everyone in this audience is get our innovative minds focussed on the full set of problems.”

How? Anne encouraged people to get involved by attending public hearings that the NSA holds.

We’ve heard time and again that even our senators and representatives don’t have access to enough information to inform NSA policies, or even be aware of what those policies are. How can we take an invitation to a public hearing seriously?

The seminar ended with a fun question and answer segment. One of the questions:

What is the NSA doing to make the scope of its data collection efforts as transparent as possible, while still achieving its goals w.r.t. national security?

The room erupted in laughter when Anne replied that the FOIA (Freedom of Information Act) was a solution to this.

⦿

Earth's Water is Older Than the Sun

Astronomers have discovered that much of the water on Earth—and the solar system—predates the Sun.

This isn’t really surprising, right? I mean, it lines right up with the Genesis creation. Still, thinking about things like this blows my mind.

1 In the beginning God created the heaven and the earth. 2 And the earth was without form, and void; and darkness was upon the face of the deep. And the Spirit of God moved upon the face of the waters. 3 And God said, Let there be light: and there was light. 1

Genesis 1:1-3 (KJV)


  1. I’m no scholar but I read this as, “Let there be light: and there was the Sun.”
⦿

Starting Sentences With I.E. or E.G.

A parenthetical statement that is a complete sentence should begin with a capital.

I used to debate this everytime I’d use i.e. or e.g. in my writing. Not anymore!

Right way:

E.g., this is a proper example.

I.e., something like this is correct.

Wrong way: 1

e.g., This is a bad example.

Apparently it is also good practice to include the comma.


  1. Sadly this is how I’ve usually written it, except I almost always left off the comma.
⦿

Hello World

Welcome to my new blog! 1 All of my blogging in the past has been in the form of longer articles, mostly technology related, and mostly trying to teach a tip or trick.

I’m taking a different approach this time. I’ll be posting more random thoughts. Sharing articles I find interesting. Overall I expect this to feel more like reading my Facebook wall and that’s what I’m going for. 2

Here goes nothing!

Hello World! Hallo Welt! שלום עולם

  1. This is no longer the first post as I’ve started migrating older content from my blog at zduck.com.
  2. This is my plan to take control of my public content. I’ll still share on Facebook and Twitter, but I want to own the canonical version.
⦿

Go 101: String or Byte Slice?

One of the first things you’ll notice in Go is that two different types are commonly used for representing text. string and []byte. A quick example is the regexp package which has functions for both string and []byte.

What is the difference?

string is immutable and []byte is mutable. Both can contain arbitrary bytes.

The name “string” implies unicode text but this is not enforced. Operating on string is like operating on []byte. You are working with bytes not characters.

They are nearly identical and differ only in mutability. The strings and bytes packages are nearly identical apart from the type that they use.

Q: If strings are just arbitrary bytes, then how do you work with characters?

A: What you are thinking of as a character, Go calls a rune. One way to iterate the characters in a string is to use the for...range loop. Range will parse the string as UTF-8 and iterate the runes. Read the for loop section of Effective Go for more information.

When to use string?

Ask not when to use string but rather, when to use []byte. Always start with string and switch to []byte when justified.

When to use []byte?

Use []byte when you need to make many changes to a string. Since string is immutable, any change will allocate a new string. You can get better performance by using []byte and avoiding the allocations.

C# perspective: []byte is to System.StringBuilder as string is to [System.String][systemstring] when it comes to performance.

Even if your code isn’t directly manipulating the string, you may want to use []byte if you are using packages which require it so you can avoid the conversion.

Converting to and from []byte is easy. Just remember that each conversion creates a copy of the value.

s := "some string"
b := []byte(s) // convert string -> []byte
s2 := string(b) // convert []byte -> string

Converting to/from string and []byte copies the entire value. Using lots of type conversions in your code is typically a warning sign that you need to reevaluate the types you are using. You want to minimize conversions both for performance and clean code.

More about strings

The Go blog has posted in detail about strings, bytes, runes, and characters in Go. You should definitely read that post to fully understand the topic.

Update: Thanks to @mholt6 for reviewing the post and helping improve it!

[systemstring]: http://msdn.microsoft.com/en-us/library/system.string(v=vs.110).aspx

⦿

Go 101: Methods on Pointers vs. Values

Methods can be declared on both pointers and values. The difference is subtle but important.

type Person struct {
     age int
}

// Method's receiver is the value, `Person`.
func (p Person) Age() int {
     return p.age
}

// Method's receiver is a pointer, `*Person`.
func (p *Person) SetAge(age int) {
     p.age = age
}

In reality, you’d only define getter and setter functions like this if you needed to implement additional logic. In an example like this you’d just make the Age field public.

This is how you define getter and setter functions in Go. Notice that we defined the Age() function on the value but SetAge() on the pointer (i.e. *Person). This is important.

Go always passes by value. Function parameters are always passed by copying them as opposed to passing a reference. (Read more.)

Even pointers are technically passed by value. The memory address is copied, the value it points to is not copied.

Here is the wrong way to define SetAge. Let’s see what happens.

func (p Person) SetAge(age int) {
     p.age = age
}

p := Person{}
p.SetAge(10)
fmt.Printf("Age: %v", p.Age()) // Age: 0

▶ Run it.

Notice that the output is 0 instead of 10? This is ‘pass by value’ in action.

Calling p.SetAge(10) passes a copy of p to the SetAge function. SetAge sets the age property on the copy of p that it received which is discarded after the function returns.

Now let’s do it the right way.

func (p *Person) SetAge(age int) {
     p.age = age
}

p := Person{}
p.SetAge(10)
fmt.Printf("Age: %v", p.Age()) // Age: 10

▶ Run it.

My rule of thumb is this: declare the method on the pointer unless your struct is such that you don’t use pointers to it.

Two reasons:

  1. Performance. Calling a method on a pointer will almost always be faster than copying the value. There may be cases wear the copy is faster but those are edge case.
  2. Consistency. It is common for at least one of your methods to need a pointer receiver and if any of the type’s methods are on the pointer then they all should be. This recommendation is direct from the FAQ.

Read the FAQ “Should I define methods on values or pointers?” for more insight.

Update: Thanks to the fine folks on reddit for suggesting some improvements.
Join the discussion on reddit!

Update 2: Here are even more rules of thumb to help you choose whether to use a value or pointer receiver.

⦿

Go 101: Constructors and Overloads

Go doesn’t have constructors in the traditional sense. The convention is to make the zero value useful whenever possible.

type Person struct {
     Age int
}

// These are equivalent.
// `p1` and `p2` are initialized to the zero value of Person.
// Neither of these are nil.
var p1 Person // type Person
p2 := Person{} // type Person

// You could also use `new` to allocate which returns a pointer
p3 := new(Person) // type *Person

It is most common to use the struct initializer. e.g. p := Person{} or p := &Person{} if you need the pointer.

Sometimes you want special initialization logic. If your type is named Person then the convention would be create a function named NewPerson that returns a pointer to an initialized Person type.

func NewPerson(int age) *Person {
     p := Person{age}
     return &p
}

myPerson := NewPerson(10) // type *Person

Multiple constructors can be implemented by having multiple initializer functions. Go doesn’t support function overloads so you will need to name your functions intelligently.

import "time"

func NewPersonAge(int age) *Person {
     p := Person{age}
     return &p
}

func NewPersonBirthYear(int birthYear) *Person {
     p := Person{time.Now().Year() - birthYear}
     return &p
}

Read more in Effective Go.

Update: Thanks to Joe Shaw for the comments! I’ve updated the article with his suggestions.

⦿

PowerShell Script Module Boilerplate

One of the things I always look for when getting familiar with a new language or environment is examples of the physical file structure and logical organization of the code. How do you layout your project files? What is idiomatic?

Unsurprisingly, this isn’t always as simple as finding a few open-source projects on GitHub to reference. Believe it or not, there are a lot of pretty unorganized coders out there. I admit to being a bit OCD with my project structures. I like them clean, organized, and consistent.

In this post I’m going to cover my preferred boilerplate for PowerShell Script Modules.

Script Modules are about as simple as it gets. Typically you have one or more PS1 or PSM1 files that contain your module’s cmdlets. Beyond that you should have a PSD1 manifest.

Fork this!

This entire boilerplate is on GitHub. If you just want a solid starting point, download this repo. If you want to know more, keep reading.

File Structure

+- src/
| +- source_file.ps1
| +- ...
+- tools/
| +- release.ps1
+- LICENSE
+- README.md
  • src/ contains the PS1, PSM1, PS1XML, and any other source files for the module.
  • tools/ is where I put any meta scripts for the project. Usually there is just one script here that builds a release version of my module.
  • LICENSE - if your module is open-source, always specify what license you are releasing it under.
  • README.md - always have a README file. Even if it is only a one sentence description. Markdown is a great format to use for this.

Explicit Exports

By default, PowerShell will export all of the functions in your module. I recommend being explicit about this and always specifying which functions should be publically exported. This way it is easy to add private helper functions that are internal to your module without worrying about them being made public accidentally.

You do this by calling Export-ModuleMember at the bottom of your source file. If you have a source.psm1 file that contains a Show-Calendar function that should be public, you would do something like this:

# Show-Calendar.ps1
#
# Show-Calendar will be public.
# Any other functions in this file will be private.

function Show-Calendar {
    # ...
}

Export-ModuleMember -Function Show-Calendar

Full example →

Release Script

Every project should have a release script. You should never be manually building your distributable release. At a minimum my release script will:

  1. Generate the PSD1 manifest for my module.
  2. Save the manifest into a temporary ./dist folder.
  3. Copy all of the module source files into ./dist.
  4. Add all of the module’s source files to a ZIP file ready for me to distribute.

Here is what a simple release.ps1 script might look like.

[View on GitHub →][release_script]

<#
.SYNOPSIS
    Generates a manifest for the module
    and bundles all of the module source files
    and manifest into a distributable ZIP file.
#>

[CmdletBinding()]
param(
    [Parameter(Mandatory = $true)]
    [version]$ModuleVersion
)

$ErrorActionPreference = "Stop"

Write-Host "Building release for v$moduleVersion"

$scriptPath = Split-Path -LiteralPath $(if ($PSVersionTable.PSVersion.Major -ge 3) { $PSCommandPath } else { & { $MyInvocation.ScriptName } })

$src = (Join-Path (Split-Path $scriptPath) 'src')
$dist = (Join-Path (Split-Path $scriptPath) 'dist')
if (Test-Path $dist) {
    Remove-Item $dist -Force -Recurse
}
New-Item $dist -ItemType Directory | Out-Null

Write-Host "Creating module manifest..."

$manifestFileName = Join-Path $dist 'YourModule.psd1'

# TODO: Tweak the manifest to fit your module's needs.
New-ModuleManifest `
    -Path $manifestFileName `
    -ModuleVersion $ModuleVersion `
    -Guid fe524c79-95a6-4d02-8e15-30dddeb8c874 `
    -Author 'Your Name' `
    -CompanyName 'Your Company' `
    -Copyright '(c) $((Get-Date).Year) Your Company. All rights reserved.' `
    -Description 'Description of your module.' `
    -PowerShellVersion '3.0' `
    -DotNetFrameworkVersion '4.5' `
    -NestedModules (Get-ChildItem $src -Exclude *.psd1 | % { $_.Name })

Write-Host "Creating release archive..."

# Copy the distributable files to the dist folder.
Copy-Item -Path "$src\*" `
          -Destination $dist `
          -Recurse

# Requires .NET 4.5
[Reflection.Assembly]::LoadWithPartialName("System.IO.Compression.FileSystem") | Out-Null

$zipFileName = Join-Path ([System.IO.Path]::GetDirectoryName($dist)) "$([System.IO.Path]::GetFileNameWithoutExtension($manifestFileName))-$ModuleVersion.zip"

# Overwrite the ZIP if it already already exists.
if (Test-Path $zipFileName) {
    Remove-Item $zipFileName -Force
}

$compressionLevel = [System.IO.Compression.CompressionLevel]::Optimal
$includeBaseDirectory = $false
[System.IO.Compression.ZipFile]::CreateFromDirectory($dist, $zipFileName, $compressionLevel, $includeBaseDirectory)

Move-Item $zipFileName $dist -Force

Version Control

Always exclude your ./dist folder from source control. As a rule of thumb, you never want to store the build output of any project in source control.

Depending on how you plan to release your module, you may prefer to exclude *.psd1 manifest files from source control. This just keeps things clean and enforces that you use your release script to build the distributable.

Good Examples

Here are a few open-source PowerShell modules that I’ve found to be good examples to follow.

[release_script]: https://github.com/jpoehls/powershell-script-module-boilerplate/blob/master/tools/release.ps1

⦿

PowerShell Script Boilerplate

This post is as much for me as it is for you. I write a lot of PowerShell scripts and they tend to follow a certain pattern. This is my personal boilerplate for PowerShell scripts.

Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"

$scriptDir = Split-Path -LiteralPath $(if ($PSVersionTable.PSVersion.Major -ge 3) { $PSCommandPath } else { & { $MyInvocation.ScriptName } })

$stopwatch = [System.Diagnostics.Stopwatch]::StartNew()

try
{
    # TODO: Insert script here.
}
finally
{
    Write-Output "Done! $($stopwatch.Elapsed)"
}

What’s going on here?

  • Enables StrictMode to enforce some best practices. 1
    • Prevents use of variables that have not been initialized.
    • Cannot call non-existent properties on objects.
    • Disallows calling a function like a method, e.g., Do-Something(1 2) instead of Do-Something 1 2.
    • Prohibits creating variables without a name.
  • Sets $ErrorActionPreference so that unhandled exceptions will halt the script execution. By default, PowerShell will roll on when an exceptions is thrown and this statement makes my scripts safe by default.
  • Set $scriptDir to the directory path of the current script. This may be different than the working directory.
  • Most of the time I want to know how long my script takes to run so I include a $stopwatch that will output the elapsed time when the script finishes.
  • The try...finally ensures that the elapsed time will output even if the script throws an exception.

Need to wrap your PowerShells script with a batch file? Grab my PowerShell batch file wrapper.


  1. Read more about this on the “Hey, Scripting Guy!” blog.
⦿