Migrating from OSX keychain to Unix pass

For some time now I’ve been wanting to move away from using Apple products so that I can make a smoother transition to using another primary OS. Most of my reasoning is security and politics but I particularly find security to be a weak player on how keychain access is stored and managed on OS X. There are more than a few articles explaining how Apple fails at security especially with their iCloud services. There are a few roadblocks that stand in the way and one is the massive password/credentials export that needs to happen to make migration a reality. For me using a service for password management is not an idea I’m not fond of as services come and go oftentimes with their own stipulations and drawbacks. I could also save all of my password data in the browser and use that but thats as bad or worse as Apples security practices and though convenient it’s also not a safe solution either and is frequently prone to browser plugin hacks and lackluster security. I wanted something that I could maintain full control over that was battle tested and open source. But that would also transition seamlessly between multiple machines with ease. Enter pass a simple management solution that I found that will do everything that I want. Theres just one roadblock I had to get around to make it work. And that was exporting all of my sensitive data from keychain and getting it into pass in a usable format. There are a few useful guides and scripts on migrating to pass but nothing specifically for OS X users moving off of keychain. So, what’s left for me to do other than write my own? It doesn’t have to be great it just has to work.

Getting everything out of keychain

The first step was to get all of my data off of keychain but the GUI didn’t want to cooperate and wouldn’t let me export even a single item so back to good ol’ cli to make things happen. Luckily there is the security command baked into OS X that could do the job which offers some really nice features to interact with pwds and other items in the keychain.

To start the export I entered this command to dump the keychain items into a plaintext file. Which unfortunately brings up a prompt where you have to allow access to each item in the keychain. There are some apple scripts out there that should do the job but they didn’t work from the start for me so I just used automator to create a quick task and click through the prompts.

1
security dump-keychain -d login.keychain > keychain

Now I had all of my passwords out there in plaintext and for each item that was exported I got something like this for each of them.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
keychain: "/Users/you/Library/Keychains/login.keychain"
version: 256
class: "genp"
attributes:
0x00000007 <blob>="somesite.com"
0x00000008 <blob>=<NULL>
"acct"<blob>="username"
"cdat"<timedate>=0x32303132303130353137323635325A00 "20120105172652Z\000"
"crtr"<uint32>=<NULL>
"cusi"<sint32>=<NULL>
"desc"<blob>=<NULL>
"gena"<blob>=<NULL>
"icmt"<blob>=0x53697465735C2D5369746537 "Sites\134-Site7"
"invi"<sint32>=<NULL>
"mdat"<timedate>=0x32303132303130353137323635325A00 "20120105172652Z\000"
"nega"<sint32>=<NULL>
"prot"<blob>=<NULL>
"scrp"<sint32>=<NULL>
"svce"<blob>="some-service-name"
"type"<uint32>=<NULL>
data:
"probablyThePWOrCert"

Converting keychain items to csv

Thats cool and all but what a crappy format. I needed to convert this format to something a little more tangible for pass to import. I thought CSV would probably be a good format thats widely used and easy to parse so I tracked down a useful script to do just that written in Ruby.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
#!/usr/bin/env ruby
#
# Usage:
# security dump-keychain -d login.keychain > keychain_logins.txt
# # Lots of clicking 'Always Allow', or just 'Allow', until it's done...
# curl -O curl -O https://raw.github.com/gist/1224792/06fff24412311714ad6534ab700a7d603c0a56c9/keychain.rb
# chmod a+x ./keychain.rb
# ./keychain.rb keychain_logins.txt | sort > logins.csv
#
# Then import logins.csv in 1Password using the format:
# Title, URL/Location, Username, Password
# Remember to check 'Fields are quoted', and the Delimiter character of 'Comma'.
require 'date'
class KeychainEntry
attr_accessor :fields
def initialize(keychain)
last_key = nil
@fields = {}
data = nil
aggregate = nil
lines = keychain.split("\n")
lines.each do |line|
# Everything after the 'data:' statement is data.
if data != nil
data << line
elsif aggregate != nil
if ( line[0] == 32 || line[0] == " " )
keyvalue = line.split('=', 2).collect { |kv| kv.strip }
aggregate[keyvalue.first] = keyvalue.last
else
@fields[last_key] = aggregate
aggregate = nil
end
end
if aggregate == nil
parts = line.split(':').collect { |piece| piece.strip }
if parts.length > 1
@fields[parts.first] = parts.last
else
last_key = parts.first
data = [] if parts.first == "data"
aggregate = {}
end
end
end
@fields["data"] = data.join(" ") if data
end
end
def q(string)
"\"#{string}\""
end
def process_entry(entry_string)
entry = KeychainEntry.new(entry_string)
if entry.fields['class'] == '"inet"' && ['"form"', '"dflt"'].include?(entry.fields['attributes']['"atyp"<blob>'])
site = entry.fields['attributes']['"srvr"<blob>'].to_s.gsub!('"', '')
path = entry.fields['attributes']['"path"<blob>'].to_s.gsub!('"', '')
proto= entry.fields['attributes']['"ptcl"<uint32>'].to_s.gsub!('"', '')
proto.to_s.gsub!('htps', 'https');
user = entry.fields['attributes']['"acct"<blob>'].to_s.gsub!('"', '')
#user = entry.fields['attributes']['0x00000007 <blob>'].gsub!('"', '')
date_string = entry.fields['attributes']['"mdat"<timedate>'].to_s.gsub(/0x[^ ]+[ ]+/, '').to_s.gsub!('"', '')
date = DateTime.parse(date_string)
pass = entry.fields['data'][1..-2]
path = '' if path == '<NULL>'
url = "#{proto}://#{site}#{path}"
#puts "\"#{site}\",\"#{url}\",\"#{user}\",\"#{pass}\",\"#{date}\""
puts "#{site},#{url},#{user},#{pass},#{date}"
#puts "#{user}, #{pass}, #{date}"
end
end
accum = ''
ARGF.each_line do |line|
if line =~ /^keychain: /
unless accum.empty?
process_entry(accum)
accum = ''
end
end
accum += line
end

This beautiful thing goes through the keychain items exported from the security dump command and converts that to a new file in csv format. I ran it something like

1
ruby keychain.rb keychain.txt | sort > keychain.csv

Importing items into pass

Finally the moment of truth where we get to import things into pass and shutdown keychain for good. So far I haven’t invested much time into this and thats good because I don’t have time to write a whole library to export/import this stuff. Assuming that the last script was successful in converting these items its time to import them with a simple bash script to parse the csv.

1
2
3
4
5
6
7
8
9
INPUT=keychain.csv
OLDIFS=$IFS
IFS=,
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read flname url uname pass time
do
printf "$pass\n$username\n$url" | pass insert -m Imported-final/$flname-$uname
done < $INPUT
IFS=$OLDIFS

Everything worked out fine even with hacking together a few scripts and I had all of my items in pass in less than a half hour. As a note the conversion didn’t get me anything from keychain except for internet passwords which is all I cared about.