This isn’t going to be an all out guide on OSX to Linux. I’ve actually used both on 2 separate laptops for a long time. This is more about some tweaks to get used to with Apple wireless hardware and and Linux.

Backstory: A couple days ago my house was robbed. No one was home, no one was hurt. They did steal all our laptops though. I used the iCloud feature to set it to email me when the mac is turned on and wipe the machine. However, I am assuming it will never be connected to the internet to do that without it being somehow disabled first.

I also have of course gone and changed all passwords. For once I’m glad I use Google docs as most of my private information like bills spreadsheet (not passwords, statistics) and such are there. I actually keep little on my computers. Both laptops have the code for unscatter.com and chatfor.us on there though. Not too worried about that either honestly.

Anyway… all this happened a couple weeks after my bathtub cracked and I have had to gut the entire bathroom. So all my money has gone into materials and tools for a bathroom remodel I never wanted to do. So, in an effort to save money the replacement laptop I purchased was not a Mac.

The thieves did not take my Apple bluetooth keyboard or trackpad though. So, down to that now. I am setting up Ubuntu 12.04 32bit version. I use 32bit because of my works Juniper VPN appliance.

Keyboard: Bluetooth sync worked fine out the box. I was able to pair using the security code. Then it didn’t work. I used the bluetooth -> keyboard menu to disconnect and connect. Then it worked fine. Looks like I may need to do this from sleep as well. Also, no delete key. Delete works like backspace. Google around and I’m sure you’ll want to jump up and tell me, “hey dummy, fn + del = delete” and I’ll tell you, not with the most recent keyboard and ubuntu 12.04. As far as I can tell the fn key isn’t recognized at all. Hopefully someone in the Linux/Ubuntu community comes up with a solution for that. I did read one guide suggesting to map the F12 key to delete. I’m considering it.

One other frustrating thing is that ctrl-shift isn’t very comfortable on this keyboard. However it’s used a lot with gnome-terminal.

Trackpad: Works great! Note, one thing you’ll need to get used to going from OSX to Linux. Bottom right corner is not right click anymore, however you can two finger click to get a right click out of it.

It’s been on my list to look at Chef and Puppet for implementation where I work for a long time. It’s been of those things I keep trying to work down my priority list to get to, and just haven’t been able to have the time to focus on it enough.

Thanks to a licensing issue that put my primary project on hold, I finally got the time to look at the two. Boy, did I need the time too. I installed puppet and played with it a few months ago. This time I finally got a chance to install Chef and give it a go.

The basic conclusion I came to is the same as I believe others have mentioned. Puppet will appeal to the sysadmin focused individuals, Chef to the ones with a more developer mindset. Before I go any further, my available budget for this project would not allow me to pursue enterprise offerings for either product. That budget is $0.

Puppet really caters to the “get stuff done” mentality. Installing it is simple for our Redhat and Windows servers. As we can do the server using yum, it could be kept up to date as a part of our regular systems patching. Since we can use yum to install the client, same thing there.

Puppet’s configuration though left something to be desired. OK, knowing what you can do with Chef left something to be desired. My initial implementation is going to be to manage hosts files across servers in a network where DNS isn’t an option. The catch being several of the servers need their hostname as 127.0.0.1, which suddenly makes managing host files with a config management system a bit more difficult. Puppet doesn’t even do well with ipv4 vs ipv6 localhost definitions - http://projects.puppetlabs.com/issues/10704

Not saying this is impossible with Puppet, it just was easier to find a way to do it with Chef - http://blog.bitfluent.com/post/196658820/using-chef-server-indexes-as-a-simple-dns. That gets lazy sysadmin points.

That’s not saying the grass is really all that much greener in Chef’s yard. When I finally started diving into it I got dependency shock. So you install ruby and then ruby gems and then use ruby gems to install chef solo which you then use to install chef server which includes rabbitmq and solr. Wow, so you got ruby, java and erlang all bundled in there. Oh, and you’ll want apache because you’ll have chef-server running on localhost and use apache to proxy it to provide ssl.

Yea, Chef is pretty thick. There were also all kinds of warning about running it on RHEL6. Though, in my case my install didn’t have any issue I couldn’t figure out. Was not as clear cut as Puppet though.

In the end, I couldn’t decide between the two. I saw the merits of both. So I packaged an even more detailed comparison than above in an email to my team. We decided to go with Chef. The primary reason was the config language being ruby. Not that we’re a ruby shop, I’m the only one out of us that’s ever touched it. We decided that if we have to learn something new, learning ruby is something we might be able to use outside of the config management system as well. The upfront challenges in documenting a good install plan and such also were not considered deal breakers, they knew when I was done any of them would be able to follow me on it because we’re a team that documents. Finally the perception we got is that Chef seemed a bit better on Windows than Puppet.

So, I’m already working on a consistent install for Chef. I’ll cover that in a post later. The high level over view is since we’re not a ruby shop I’ve been able to research ways to best manage it from that level up. I’ve found rvm and it looks like I may use that to manage ruby and rubygems. Have to admit, ruby is turning out to be a pleasure to work with. I have to find where I put that Ruby Cookbook I bought used last year.

One thing I’ve noticed on my RedHat Enterprise 6 servers is telnet isn’t installed by default any more. I didn’t notice this because I use telnet to connect to server shells anymore. Port 23 hasn’t been an open port on servers I maintain in years. I presume most everyone else in the same boat and if you aren’t, umm really, you know telnet traffic is plain text right?

Now, I did use telnet back in the day for connecting to muds. However, really the only use case I’ve had for telnet in the past 10 years has been http. While wget and curl are great, I’ve found it’s just easier to do something like:


telnet www.google.com 80
Trying 173.194.73.147...
Connected to www.l.google.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.google.com
Connection: close

As stated above, by default this doesn’t work on Redhat ES6 servers. Instead of installing telnet, I’ve been installing nc. nc, otherwise know as netcat, is a much more useful tool to learn.

http://netcat.sourceforge.net/

On Redhat systems, install it using yum:


yum install nc


A good quick guide for basics of it can be found here: http://www.thegeekstuff.com/2012/04/nc-command-examples/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+TheGeekStuff+%28The+Geek+Stuff%29

That article is in fact what’s prompted me to write this post. The eight practical uses presented in that article are just scraping the surface of what you can do with netcat. In my case, it makes a good telnet replacement.


nc -v www.google.org 80
Connection to www.google.org 80 port [tcp/www] succeeded!
GET / HTTP/1.1
Host: www.google.com
Connection: close

I use the -v flag to get the validation message that the connection attempt succeeded. It’s not necessary, I just like the notification. I wouldn’t go so far as to alias it or anything, you may not want it to be as verbose in all cases.

So now you have a nice telnet replacement that you can also use for a lot of different purposes. Here’s a beginners guide I just looked up for more security minded uses of netcat: http://nrupentheking.blogspot.com/2011/03/netcat-beginners-tutorial.html

As an extra bonus, some of you may be wondering how I check out sites that require ssl. That’s where openssl s_client comes in handy.


openssl s_client -connect www.google.com:443
[ cut out a bunch of ssl connection information ]
GET / HTTP/1.1
Host: www.google.com
Connection: close

So there you go, nc and openssl along with a understanding of basic http request headers will get you a long way in debugging http.

If your http debugging requires you to start validating sites and their rendered state which may be based on css and javascript I suggest you also check out phantomjs and casperjs. Full access to a scriptable headless browser which can generate .png images of any page.

http://phantomjs.org/
http://casperjs.org/

So, after praising the virtues of Cinnamon Desktop not that long ago, I’ve switched. Why? They removed the feature I used the most and it’s gnome3 base didn’t let it run on my workstation at work. Ah the fun of an ATI card.

The feature I’m talking about. The handy
- move mouse to left corner
- drag window to new desktop

I guess Cinnamon is going down the path of trying to get Compiz working. Best of luck to them, maybe I’ll check it out again later when it matures more and I can get away from ATI.

So, why e17?

1) I’ve always loved enlightenment. I like eye candy, what can I say.
2) Performance. What’s that? Enlightenment and performance? Yes! e17 runs quicker than plasma or gnome3, period.

A taste of KDE?

Ok, the short story is I installed e17 on top of Linux Mint KDE 12 install. I like it because I can keep using Dolphin and Konsole. At work I did this by adding the Bodhi Linux repo. At home as I was on 64 bit architecture I ended up with enlightenment compiled from source.

Lesson learned here: Start with a KDE base and install e17 on top of it. You can continue to use the awesome KDE apps without the slow plasma. This is especially true on hardware where you don’t have access to great 3D support. This happens to both my ATI workstation and my m11x r1 laptop at home that I choose not turn the nvidia display on.

I’m really looking forward to Bodhi Linux to be released on top of Ubuntu 12.04. The current version doesn’t play with my laptop at home, however I’m confident the 12.04 base will be fine.

The environment I support is mixed. We have a pretty good mix of RedHat Linux Server and Windows. We’re fortunate that most of our Windows support is for Microsoft products of which we use most of the big ones. As you can guess, we do use Exchange and Active Directory.

Without getting too detailed on the architecture of my work network, let’s just say that there are times when we need to get data from Active Directory on machines that are not even Windows Server OS’s, let alone in the domain.

In one case we have a pretty complex process that supported several needs, from HR to offline Outlook files when we ran Outlook 2003 to mail lists for our spam quarantine servers to us. The process is slowly being dismantled, and the export files from Active Directory have always been a lot more complicated than what most systems needed them to be.

I recently took on removing the quarantine servers dependency on that platform. My goal was to do a lookup directly from those machines to Active Directory on a regular basis. This was sparked by the fact that the other scripts, which used ldapsearch, broke when I tried to migrate them from ES5 to ES6. The issue was the openldap version changed and the output for SimplePageResults changed between the openldap versions.

This caused me great concern and I decided that the better solution was to rewrite something entirely new to make the ldap queries with, like say.. python! As you might guess, I’m a pretty big fan of python. Python was the wrong choice.

I wrote a script, using python-ldap that worked. Well, it worked until I tried it on a 64bit machine, then it didn’t work. As far as I can find, it’s not going to work on 64bit Linux either. Also, I still had it only talking to one domain controller. If that server was down the script would break. I spent about 2-3 hours on that script only to run into the 64bit issue.

This prompted me to look back at the original script. It then struck me, if I only retrieved the mail attribute in the ldap query, I could “grep ^mail: ” and not get any of the other noise that openldap adds in ES6. Also, ldapsearch supports multiple hosts, so you can do something like

ldapsearch -H “ldaps://dc1.domain.com,dc2.domain.com,dc3.domain.com” …

Paging? ldapsearch does that, you can tell one command to page untli it hits the last one and spit out all the output. Simple as pie.

It took me 30 minutes to write a script that included instructions for setup and use, variables for most dynamic items and spit out clean consistent output. Lesson learned, bash + ldapsearch is the way to go. In the future if I had to do ldap queries from a script, I’ll run the command as a system call if I have to. Much easier to manage. Sometimes I need to remember as much as it is to program, my job is being a Systems Administrator. That means using tools instead of writing them all the time.

So today this little link popped up on my Twitter feed.

http://bugs.ruby-lang.org/issues/4996

It’s a discussion that ran 8 - 5 months ago about when to stop supporting Ruby version 1.8.7. The reason for this blog post is this quote: “So why there’s still 1.8.7? It’s also clear: for system admins.”

It then goes on to discuss how it probably makes sense at this time to start considering when to set a real end of life date for the version. Now sysadmins have been catching a lot of flack lately. Business people think we say no too much. Developers accuse us of slowing them down.

But, seriously, you think sysadmins are also interested in keeping developers working on a version of ruby that’s years old? No, no and I repeat no. Let me explain how this really works.

Developer builds something in Ruby 1.8.
It goes live.
Developer moves on to a new project.
Sysadmins are now keeping the product live.
OS versions get upgraded.
Sysadmin checks, Ruby 1.8.7 is still supported. OK, no need to call out a need to update that app.

Why does this happen? When the sysadmin does come up and say hey, you need to update your app to a new version of Ruby there’s usually much gnashing of teeth. The developer has to stop new project work to upgrade their code. The business user has their list of stuff they’re waiting on delayed because the developer is busy maintaining legacy code.

This isn’t the sysadmins fault. We just need to update our servers because we can’t run them on unsupported versions of our OS.

You know what we like? We like EOL dates. We like to know them well out in advance. We watch for these things. We have a list of what apps we’re currently running and what version of software they’re on. We also keep checking what the current release is at and when the version we’re running is going to reach EOL. We even keep checking these things after Ruby becomes less popular and all the devs and burbling happily about node.js. When we see that EOL date we notify everyone as soon as possible. That gives the business users, project managers and developers time to schedule their upgrade without it being a fire drill.

No, Aaron Patterson (that’s who it says authored that post), it’s not the sysadmins fault you’ve been maintaining ruby 1.8.7 for three years. It’s not the sysadmins fault people have continued to develop applications on it, steadily increasing the amount of time and pain it will take to get off of 1.8.7. The reason people are still using 1.8.7 these 3 years is because you chose to support it instead of setting an EOL for it to force people to upgrade sooner.

Ah Gnome3, KDE 4.x, Unity… how the GUI desktop landscape has changed over the past year or so for Linux. Honestly, I haven’t been a fan of any of them.

KDE, it’s nice but experience has been it just seems slow. I’m not sure how to explain it, but just basic interaction with desktop has always left me with this feeling of it being sluggish. Once upon a time I was a huge KDE fan, likely from my years of using Slackware for everything. I actually did just install it at work, the ATI card I have there doesn’t play nice with Gnome3, therefore Cinnamon as well.

Gnome3, just feels like a mess. A vanilla Gnome3 install just isn’t very intuitive. Gnome has a trend of trying radical things that are just frustrating. Remember when Nautilus went spatial? Ugh. Drove me nuts a time I was actually turning into a huge fan of Dropline Gnome. With Gnome3 was it really necessary to for ctrl-alt up/down instead of left/right for desktop switching? I only have over a decade of muscle memory there.

Unity… it’s interesting. I think a big reason I tend to stay away from it is my negative experiences with it when it first came out. It was glitchy on my hardware, locking up at random times. In all fairness with 11.10 it worked fine. I think I just had developed a bad taste for it by then. Unity has no problem running on an ATI card though *nudge Gnome3*.

Now Cinnamon. The only fault I can find with it is it’s Gnome3 roots means it has the same hardware issues. My laptop with an Nvidia card runs it great though. Cinnamon just seems to be what Gnome3 should have been.

Cinnamon has lower panel with the application menu. I like the fact you can add apps to it real easy, acting like a dock. I have a Macbook I use primarily for working from home, I’m used to having a dock (and at the bottom of the screen, Unity). However it builds on Gnome3’s always having one empty virtual desktop. Over all I find the virtual desktop management which is pretty much all Gnome3 to be an improvement. It’s real easy to move applications from desktop to desktop. When visually managing the desktops it’s vertival, however Cinnamon kept the left/right key bindings and visual transitions. I don’t have to relearn anything.

Over all, it’s appearance is a little rough, but the functionality has been a perfect fit. About the only improvement I’d make is a top panel that functions like on Unity and my Mac, holding the current application context menus and such. I just prefer that approach.

If you are running Mint and haven’t tried it yet, sudo apt-get install cinnamon-session and give it a go. It’s been the most intuitively easy to use desktop released this year.

So for my current project ChatFor.Us I’m using asyncmongo with tornado. I’ve started messing around with tornado.gen to try and get rid of the spaghetti callbacks I was building up. First of all, I love tornado.gen, it’s great stuff.

asyncmongo is a special case, in the it doesn’t return just one item to callbacks. Rather, it returns {response, error=None}, or error can be an object if there is an actual error. This gets a little tricky. Turns out I wasn’t the first person to run into this.

https://github.com/facebook/tornado/issues/351

As you can see, the issue has been resolved, but no one really explained how it works anywhere. What Ben did is basically build an Arguments object which is what gets returned. You can access it various ways, and here’s how I’ve found is the simplest for me.

I guess at this point code explains it best.


# execute the query, expecting args and kwargs which is a tuple, dict response
response, error = yield tornado.gen.Task(db.users.find_one, {"id", id})
# result is actually the first item in response which is a tuple
result = response[0]


I have an idea for a modification to make this a bit simpler, and will try to put together a pull request soon.

This is a follow up to my last post. Without actually writing any code I figured out a way to possibly pull of the idea in my previous post. It was a little complicated and had room to fail. It also didn’t cover keeping people logged in long term. Basically the whole “Remember me for x days” would be difficult to implement. It wouldn’t be impossible, but was complicated enough to make me think that maybe there is a better way.

Actually there is a better way, https. The problem I have with https is it costs money. Yea, not a lot, but still there is a cost associated with it and I like the whole idea of the free web. The nice thing about https is that it helps ensure you’re talking to who you think you’re talking to. That’s the whole point of ssl certificates. It’s not perfect, there have been problems in the past. However it’s a pretty good system that gets the job done.

I’m not proposing to change that. My idea is for when you need less secure data transmissions. httpe or http encrypted.

The basic principle is that requests and responses would be encrypted using a shared key between the client and server. The basic negotiation would be:

Client sends an unencrypted request, which consists of randomly generated key.
The Server responds with a response encrypted with the key sent by the browser. The payload being a new key.
The Client and Server could then begin encrypted conversations using the server generated key.

You could then set cookie headers and such as you like, for example to maintain persistent across sessions.

The risk of someone hijacking a session is limited to someone sniffing the original request. There would be some overhead. The server would need to maintain the key for at least the duration of the session. Then again, it already needs to do this for the session token any way.

At the worst this would allow people to set up and get basic sessions and encrypted traffic running for their prototype and early beta ideas without forking over the money for an ssl cert to start. Meanwhile the way we’ve been doing http sessions for years doesn’t really have to change.

Edit: HTTP 1.1 Upgrade header looks like a similar approach that’s probably a lot better than the one I came up with while drinking coffee. http://en.wikipedia.org/wiki/HTTP/1.1_Upgrade_header

First, please excuse any typos or weird formatting. Writing this post on my Android while waiting on my car to be serviced. Been here 4 hours so far, doing this also to kill boredom.

This is a post I have been wanting to do for a while. I have now written 2 sessions libraries. Gaeutilities and asyncmongo sessions. So I feel I have a pretty good grasp of the issues.

With both libraries I introduced some security concepts that affect performance. The main security issue with http sessions is unless you are using ssl it’s too simple to sniff and replay data to hijacking a users session.

My implementation both times to address this has basically been a kludge. I constantly change the token stored as a cookie in the browser to make hijacking more difficult. I also suggest people implementing the library always tie the user account to an active session.

In an ideal all this token rotation would not be necessary. With HTTP 1.1 we can keep a persistent connection open. What would be nice is if we could start using that connection as a session and also have a way to recover if that connection is lost and reestablished.

We would also need a way for load balancers to interact with backends to manage that session id. That way the lb could manage the session itself. There already exist headers for proxies to pass client up information, persistent connection sessions could key off of that.

Cookies would still be useful. Sessions are really only necessary when identity must be confirmed.

Sessions are a problem that have been solved a lot of ways by a lot of frameworks. However as we move to browsers and http servers ( or at least proxies ) supporting persistent connections I think it can be done in a way that provides more security.

This is probably a little vague. I may see if I can a hack a model together that demonstrates the concepts. The main question is how to handle the connection being lost and not losing identity when the client reconnects. I don’t think moving to requiring a constant connection only is the right answer, that feels like a step backwards as well.