Allan's Ramblings thoughts on technology and more 2012-01-27T14:10:20-08:00 allan http://www.allannienhuis.com/index.php allan@allanandshelley.ca tag:ramblings,2012:AllansRamblings Pivot Copyright (c) 2012, Authors of Allan's Ramblings Dreamhost Trac misconfiguration 2012-01-27T14:10:00-08:00 2012-01-27T14:10:00-08:00 tag:ramblings,2012:AllansRamblings.45 Dreamhost is a great hosting company, and provides a lot of very nice 'one click installs' of common software packages.  I sometimes use Trac (http://trac.edgewall.org/) for managing hobby development projects, and the Dreamhost one click install worked great, except when it came to setting up authentication (requiring login). It is simple enough to set up .htaccess and .htpasswd files based on the Trac documenation, but authorization fails for all javascript, css and related files (prompting the user with multiple login dialogs). After much searching about, I found the solution to the problem here: http://discussion.dreamhost.com/thread-124412.html

Simply put: the installer misconfigures the trac.ini file for the htdocs_location setting. Rather than using a relative path, it uses an absolute path with full domain name, which causes issues with the authentication configuration when using .htaccess files.

The solution is to change:

htdocs_location = http://www.yourdomain.com/trac/htdocs

to:

htdocs_location = /trac/htdocs

of course substitue the correct/actual path to the htdocs folder if you chose a custom name/path for your Trac install.

Works like a charm!

]]> allan Comet or Long Loop message pattern 2011-04-17T08:21:00-08:00 2011-03-26T20:12:00-08:00 tag:ramblings,2012:AllansRamblings.44 A while back I posted a possible solution to dealing with long running processes in a web application. While that solution works for very basic processes, the use of threading in an asp.net application can be the cause of a lot of grief (there are just too many ways outside of your control for those threads to be aborted prematurely). I did a little research and came up with a MUCH better solution - simply execute the ajax request for the long running process, and then listen for messages on another ajax request. The key to this working, however is to ensure that your long running process is a SESSIONLESS request, otherwise your request will block further ajax requests until it's completed. A while back I posted a possible solution to dealing with long running processes in a web application. While that solution works for very basic processes, the use of threading in an asp.net application can be the cause of a lot of grief (there are just too many ways outside of your control for those threads to be aborted prematurely).

I did a little research and came up with a MUCH better solution - simply execute the ajax request for the long running process, and then listen for messages on another ajax request. The key to this working, however is to ensure that your long running process is a SESSIONLESS request, otherwise your request will block further ajax requests until it's completed.

In my previous solution, I pointed out a number of weakneses with the simple polling technique I used to send status messaged back to the client. Polling is too chatty, especially if you wind up sending redundant status data back from the server regularly.

In this blog post, I'll focus on the improvement to the message passing portion of the solution.

As web developers we are constantly on the lookout for server requests that take 'too long' to respond, so this solution may be a little counter-intuitive for some. The basic idea is that we submit an ajax request to the server which INTENTIONALLY doesn't return a response to the client until it has new data to send back. The code on the server waits in a loop until the relevant data is available. When the client eventually receives the response, it IMMEDIATELY makes another request to the server for more data. In this way it will always have an open connection to the server, which can send data back to the client when IT is ready, not when the client decides it's interested. This simulates a 'push' of data from the server to the client, which is an extremely useful concept.

In order to have a good user experience this design does need to incorporate a bit more error handling. First, the client must handle timeout errors (default timeouts can be set when you make the ajax request, or you can accept the server defaults (typically 20 minutes). Second, the client should have a way of aborting the 'listening' process based on user input. Third, the server should handle some type of logical timeout in case it unexpectedly stops finding data to send back to the client. The method used to communicate a server-initiated timeout could also be used to communicate other logical statuses, such as 'process completed - stop listening'.

On the server

I'll show the code for the server first, because this is actually the simplest part of the solution:

As you can see, this simply loops until it has something to send back to the client. I chose to sleep the loop for .5 seconds in order to avoid consuming too many resources on the server. You can also see that I implemented a simple 1 minute timeout.

Note: in a real-world production application, you should use asynchronous controller methods, or you will risk depleting the IIS threadpool. This is just a sample app focusing on the communication pattern. Another alternative would be to use a completely different messaging server that doesn't consume threads as agressively as IIS/.net - such as Faye built on top of node.js. In this option, the .net server would send messages to the messaging server, and the web clients would 'listen' to the messaging server.

On the client

This is a simple ajax request, with handlers for errors and success. You'll notice I return the jqXHR object recieved from the ajax call. This is used later to abort the open connection based on user input. The error handler isn't that interesting - it just displays the error and stops the process.

When we recieve a message from the server, we display it and then check to see if we should continue listening for messages. In this case I have encapsulated the code into a 'listener' object which has start(), abort(), and isListening() methods, so I call the abort() method simply to reset the state of the object, not to abort the ajax request (which is completed at this point).

Here are their implementations:

And to wrap things up, here is how these are wired into the user interface:

You might notice that I have the UI code pass in a callback function which is responsible for displaying the recieved messages in the UI. This provides a nice clean separation of concerns between the message handling and the UI elements.

You can find a complete asp.net MVC3 solution here: Comet Demo Naturally this pattern could be implemented in any development platform, although an ajax library such as jQuery and an MVC framework that handles the details of dealing with json data keeps things a lot simpler.

Let me know if you have feedback on this pattern or on the code sample.

Cheers,

]]> allan Javascript videos 2011-03-20T19:28:00-08:00 2011-03-20T14:42:00-08:00 tag:ramblings,2012:AllansRamblings.43 If you are a web developer, you almost certainly need to program in Javascript.  If you need to program in Javascript, you need to watch this series of video presentations by Douglas Crockford.   Hopefully most of you (web developers) know who Mr Crockford is, but for those that don't recognize his name: he works for Yahoo, and is a well known author and presenter on Javascript topics.  He is a member of the ECMAScript standards body and a general Javascript guru - developer of JSLint, the JSON spec and author of JavaScript: The Good Parts.  These lectures were given to (some members of) the Yahoo development team.  The first lecture is a fascinating history of computing and language development which is really informative and sets up the other lectures (on Javascript) really well.  If you don't have the time for the first lecture you can dive in on the second one and get right into the language implementation details, but I really do recommend you start with the first video.  Each presentation is about 2hrs long, so make sure you've set aside enough time - it will be worth it! I can't recommend this enough - if you're serious about your professional development as a web developer, Mr. Crockford's material is must-read and must-watch. If you are a web developer, you almost certainly need to program in Javascript. If you need to program in Javascript, you need to watch this series of video presentations by Douglas Crockford.

Hopefully most of you (web developers) know who Mr Crockford is, but for those that don't recognize his name: he works for Yahoo, and is a well known author and presenter on Javascript topics. He is a member of the ECMAScript standards body and a general Javascript guru - developer of JSLint, the JSON spec and author of JavaScript: The Good Parts.

These lectures were given to (some members of) the Yahoo development team. The first lecture is a fascinating history of computing and language development which is really informative and sets up the other lectures (on Javascript) really well. If you don't have the time for the first lecture you can dive in on the second one and get right into the language implementation details, but I really do recommend you start with the first video. Each presentation is about 2hrs long, so make sure you've set aside enough time - it will be worth it!

I can't recommend this enough - if you're serious about your professional development as a web developer, Mr. Crockford's material is must-read and must-watch.


]]> allan Long Running ASP.net Processes 2010-11-28T10:17:00-08:00 2010-11-27T09:33:00-08:00 tag:ramblings,2012:AllansRamblings.42 The problem Sometimes, your web application needs to do something that takes a really long time - perhaps process a batch of files, backup or archive data, gather a bunch of data from external sources, or similar.  When dealing with this situation, you're faced with a few challenges: browser and other timeout settings - web frameworks aren't designed to take more than a few seconds long to process a request and send back a response to the user. user feedback - the user needs some sort of indication that the system is working as intended and not frozen or encountered an error. user productivity - the user may want to do something else within your app while waiting for your process to finish.I had to solve this problem myself a little while ago and thought I'd share my solution, which has a few concepts I did not find while searching for articles on the topic: Status update in the form of a log or history of process rather than just a single %age complete number used in a progress bar Providing parameters into the long running process, and Getting access to the HTTPContext during the long running process. The problem

Sometimes, your web application needs to do something that takes a really long time - perhaps process a batch of files, backup or archive data, gather a bunch of data from external sources, or similar. When dealing with this situation, you're faced with a few challenges:

  • browser and other timeout settings - web frameworks aren't designed to take more than a few seconds long to process a request and send back a response to the user.
  • user feedback - the user needs some sort of indication that the system is working as intended and not frozen or encountered an error.
  • user productivity - the user may want to do something else within your app while waiting for your process to finish.
I had to solve this problem myself a little while ago and thought I'd share my solution, which has a few concepts I did not find while searching for articles on the topic:
  • Status update in the form of a log or history of process rather than just a single %age complete number used in a progress bar
  • Providing parameters into the long running process, and
  • Getting access to the HTTPContext during the long running process.

The Solution

The key technologies used in solving the posted challenges are:

  • Ajax in the browser to dynamically update the UI with the current status in a smooth and expected manner for the user.
    • I used the excellent jQuery libraries for this behavior.
  • Threads on the server to spawn a process that continues to run after a response has been given to the user.
  • Use of the ASP.net Cache as a way of communicating between the long running process and the rest of the web application
  • Use of JSON data to pass information between the browser and the server.
I've chosen to use ASP.net MVC in my example, because it provides very easy means to work with JSON requests and responses (and the framework rocks!). However the same approach could be used with ASP.net webforms as well. The MVC framework is not core to the solution. In fact, this approach could be used in a Java application just as well, given the ability to create threads and use some means of inter-process communication (session variables or similar).
The design is pretty straightforward. Let's look at the code in the browser first.

In the Browser

I won't show the HTML markup here, but it is very simple - just a jQueryUI button for the user to click. When the user clicks the button, we display a dialog box to hold the status updates, and the process is kicked off with an ajax request.
The browser stores that processID and creates a timer which will poll the server every second for an update on the status of that process:
Obviously polling every second may not be ideal so the frequency should be adjusted to suit your needs.
When the response with the updated status returns, the UI is updated:
When the process is completed, we let the user know and trigger an optional cleanup routine on the server that I'll go into more detail later. In this example I'm throwing an alert dialog to make it obvious to the user that the process is done, but this is just an example - you might want to do something a bit more user-friendly than that in your application.
Now, let's see what's happening on the server.

On the Server

Here is the controller action that triggers the long running process and returns a unique ID:

That is pretty straightforward and not very interesting, other than to point out that the only job of this action is to trigger the long running process, not do any of the actual work. More interesting is how it triggers the process:

Here we create a new Thread to do the actual work. The 'MyLongRunningProcess' is a method that we created to do the actual work. The 'ParameterizedThreadStart' method allows us to pass a single parameter to that method, so we use a simple object to hold whatever data we want to use during that process.

There are a few interesting points about the parameters:

  1. Because we've used a simple object to hold the data passed to the process, we can pass in whatever complex information and objects we might need.
  2. I've passed in a GUID processID that the long running process can use to tag it's status updates, and
  3. I've passed in the current HTTPContext so that the long-running process can have access to the environment of the web application. This is particularly useful if you are migrating controller code that may have assumptions/dependencies on the HTTPContext.

Here is the long running process itself:

Notice that I've assigned the HTTPContext to the System.Web.HttpContext.Current property. I was actually surprised that I was able to do this. We are now keeping a handle to that Context which would otherwise be disposed of after the original request was made. I am assuming that when this thread finishes execution, that things will be disposed of during the normal .net garbage collection process, so I don't explicitly null the System.Web.HttpContext.Current at the end of the process. Perhaps I should - anyone with a bit more insight into this please leave a comment!

The process leaves status updates in the HTTPCache:

All we are doing is appending a new message onto the end of the existing status, which is simply an html string. This is very simple and actually works really well, but there are a few issues with this that I want to highlight:

  • Because of the polling design, we won't know when the user has the very last status update, so I rely on a final action triggered by the browser to clean up the cache entry. Obviously this isn't guaranteed to happen - the user might close the browser or navigate to a different page, or the network might fail and not deliver the cleanup request.
    • However, because the ASP.net cache will eventually kick the entry out of the cache based on it's expiry rules, we don't have to worry about a permanent buildup of garbage like we would if we were storing status in a file or other more permanent resource.
  • The amount of data passed from the server to the client grows with each update to the status, and the majority of that data is redundant. Depending on the polling frequency, if your status update includes lists of thousands of files or similar, this could quickly become a real performance issue.
    • This design could be refactored to only retrieve status updates that the client hasn't already recieved. Because of the inherent unreliability of http protocols, we can't just delete the status information once we send it back to the browser, we would have to have requests for status updates include some type of pointer to the last update recieved (datetime might suffice, but I think an ID of some sort would be more reliable), and send updates that have occurred after that point.
  • The example stores the status as HTML (it includes <br> tags as line separators). This is not a good separation of design/layout and content/logic.
    • An improvement would be to separate the updates with some other token (newlines), parse the data on the client, and apply whatever layout styling is appropriate, perhaps using jQuery templates.
    • A variation would be to have the updateStatus method apply some type of template rendering (HTML.RenderPartial()?) to the status before it is inserted into the cache.
  • The example uses a magic string in the returned status to determine when the process is completed. This is also not a good separation of design/layout and content/logic, and the status update could incorrectly think the process is finished if your status update includes that magic keyword before the process is finished - such as having the keyword in a filename, or similar.
    • An improvement would be to return the 'complete or not' status (or a %age complete amount) as an additional parameter in the status update response, which the browser could check. This could be stored in a separate cache element (make sure to use the processID in the key).
Here is the action used to return the status update to the browser:
And here is the action used to cleanup the cache entry when we're done:

General notes and comments

In this example, the user can trigger multiple processes by clicking the button again before existing processes are finished. Each status dialog will get it's own updates appropriately.
The user can also close the dialogs before the process is finished. This only hides the dialog. The updates continue to happen, the user just can't see them. You might need to provide a way for the user to bring the dialog back up again. Obviously the choice of a dialog in the first place is just one of convenience - you can put the status updates wherever you like.
Note - this process still runs inside the asp.net application pool, so is subject to issues like app-pool recycling for various reasons, which would terminate your long running process. If you have a _really_ long running process, or need one that is guaranteed to stay running through application restarts, you need to create a windows service, run the thread there, and communicate to and from the web application in a completely different manner - something beyond the scope of this blog post. :)
I hope this helps someone needing to do something similar in their application. Feel free to leave questions or comments!
Here is a link to the source code: Long Running Process Example

]]> allan Learning another language 2009-10-07T19:45:00-08:00 2009-10-07T19:45:00-08:00 tag:ramblings,2012:AllansRamblings.41 I've been meaning to learn another programming language as a way of 'sharpening my tools', so to speak.  For years I've had my eye on LISP, but wasn't sure it would be worth my time, given the predominance of the MS .net stack, Java, javascript, and some of the new scripting languages - ruby, python, etc.  But I stumbled across a recommendation to watch a series of videos on LISP from MIT professors Hal Abelson and Gerald Jay Sussman.  They are freely posted here: http://www.archive.org/details/mit_ocw_sicp  Well I tried the first lecture, and while I chuckled at the dress and hairstyles of the early 80's, I was quickly hooked on the content.  I quickly realized that this wasn't just a course on LISP, but rather a great course on the really great concepts of computer programming.  I'm on the 3rd lecture so far, and having a blast.  This might be old-hat for some CS grads, but I'd still really recommend checking these out if you want to geek out on some advanced ideas. I've been meaning to learn another programming language as a way of 'sharpening my tools', so to speak. For years I've had my eye on LISP, but wasn't sure it would be worth my time, given the predominance of the MS .net stack, Java, javascript, and some of the new scripting languages - ruby, python, etc. But I stumbled across a recommendation to watch a series of videos on LISP from MIT professors Hal Abelson and Gerald Jay Sussman. They are freely posted here: http://www.archive.org/details/mit_ocw_sicp Well I tried the first lecture, and while I chuckled at the dress and hairstyles of the early 80's, I was quickly hooked on the content. I quickly realized that this wasn't just a course on LISP, but rather a great course on the really great concepts of computer programming. I'm on the 3rd lecture so far, and having a blast. This might be old-hat for some CS grads, but I'd still really recommend checking these out if you want to geek out on some advanced ideas.

]]> allan Cisco VPN for Win7 x64 2010-03-22T10:15:00-08:00 2009-08-09T00:42:00-08:00 tag:ramblings,2012:AllansRamblings.40 The Cisco VPN client won't run on 64bit versions of windows, and Cisco has no plans to ship a 64 bit client anytime soon.  In order to get VPN working on my shiny new Windows 7 x64 install, I tried to get the Cygwin/linux vpnc client working.  I ran into a number of problems, so this post explains how it can be done and hopefully will help you avoid the same problems I had.  I spent a lot of time researching information on the 'net, and wound up tweaking steps a bit and hacking scripts myself, but much of this is based on work by Li Zhao and Salty.NOTE: update 22 Mar 2010 - The Shrew.net client supports Win7 now, and seems to work flawlessly. It’s a much better option if you don’t already have a need for Cygwin. To get Cisco VPN working on an x64 Windows 7, you need: Cygwin - a wonderful set of tools to emulate a linux environment on windows OpenVPN - to provide a virtual TAP network adapter VPNC - the VPN ClientHere's how to do it: The Cisco VPN client won't run on 64bit versions of windows, and Cisco has no plans to ship a 64 bit client anytime soon. In order to get VPN working on my shiny new Windows 7 x64 install, I tried to get the Cygwin/linux vpnc client working. I ran into a number of problems, so this post explains how it can be done and hopefully will help you avoid the same problems I had. I spent a lot of time researching information on the 'net, and wound up tweaking steps a bit and hacking scripts myself, but much of this is based on work by Li Zhao and Salty.

NOTE: update 22 Mar 2010 - The Shrew.net client supports Win7 now, and seems to work flawlessly. It’s a much better option if you don’t already have a need for Cygwin.


To get Cisco VPN working on an x64 Windows 7, you need:

  • Cygwin - a wonderful set of tools to emulate a linux environment on windows
  • OpenVPN - to provide a virtual TAP network adapter
  • VPNC - the VPN Client
Here's how to do it:
I turned off UAC for the purposes of getting all of this installed. I ran into a number of permissions issues and false starts when I left it turned on. This reduces the number of things that can go wrong in the install process, particularly with Cygwin. Feel free to turn UAC back on after the install process if you're so inclined.

Install Cygwin:
  • Download cygwin http://www.cygwin.com/setup.exe installer and save to your hard drive (don't just run it directly)
  • Right-click on the setup file, and 'Run as administrator'. This may not actually be neccesary but it makes me feel good.
  • Accept defaults except for local packages directory - I changed that to c:\cygwin\packages. Saving stuff to my desktop is annoying.
  • Choose a mirror to download packages from. Choose something physically close to you.
  • Choose the following additional packages to include: (choose them by clicking on the word 'skip' in the 'New' column)
    • Devel > gcc-g++: C++ Compiler
    • Devel > make: The GNU version of the 'make' utility
    • Libs > libgcrypt: A general purpose crypto library based on the code from GnuPG
    • Libs > libgcrypt-devel: A general purpose crypto library based on the code from GnuPG (development)
    • Libs > libgpg-error: A library that defines common error values for GnuPG
    • Perl > perl: Larry Wall's Practical Extraction and Reporting Language
  • The rest of the cygwin install process is pretty obvious; just a couple more mouse clicks.
Install OpenVPN:
  • Download the latest OPENVPN release candidate from http://openvpn.net/index.php/open-source/downloads.html
  • Save it to your hard drive somewhere you can find it.
  • Make sure to run the installer as an administrator and in windows vista compatibility mode: (this IS neccesary!!)
    • Open up windows explorer to that location.
    • Right click on the file you saved, and select 'properties'
      • Select the 'compatibility' tab
      • In the Compatibility mode section, check the box 'Run this program in compatibility mode for:', choose Windows Vista (service pack 2)
      • In the Privilege Level section, check the box 'Run this program as an administrator'
      • OK/Apply the settings.
  • Now run the installer. Agree to the license agreement.
  • Uncheck all of the options EXCEPT for: 'TAP Virtual Ethernet Adapter', and 'add shortcuts to Start Menu' (in case you want to add another vpn connection)
  • Install to default location.
  • The installer should have created a TAP adapter in Control PanelNetwork and InternetNetwork Connections. It's probably called Local Area Connection 2. You NEED to rename it to something without a space in the name and something you'll remember. I called mine 'CiscoVPN'.

Build and install VPNC:
  • Download the latest release of vpnc from : http://www.unix-ag.uni-kl.de/~massar/vpnc/ I got http://www.unix-ag.uni-kl.de/~massar/vpnc/vpnc-0.5.3.tar.gz
  • Save the file to a convenient temporary location, such as c:\cygwin\tmp
  • Right click on the Cygwin Bash Shell shortcut (should be on your desktop and/or start menu) and select 'Run as administrator' to open up a cygwin bash command prompt. IT IS CRITICAL THAT YOU RUN THIS AS ADMINISTRATOR OR YOU WILL FAIL AT CERTAIN POINTS IN THIS PROCESS SILENTLY!!!
  • Now extract the contents of the file and change to the extracted folder:
$ cd /tmp
$ tar xvfz vpnc<tab>
$ cd vpnc<tab>
  • Now, compile it with make:
$ make
$ make PREFIX=/usr install
$ mkdir /var/run/vpnc
  • For or some reason my own Cygwin install did not put /usr/sbin into the 'path' environment variable. Not sure why that is. You could add c:\cygwin\usrs\bin into your system path via advanced system properties in Windows, but here's how I fixed it: I edited my .bashrc file (found in your 'home' directory: /home/<username>/.bashrc ), and added these two lines:
export PATH=${PATH}:/usr/local/bin
export PATH=${PATH}:/usr/sbin/
  • You'll want to logout out of the bash shell and login again to have that take effect. REMEMBER TO 'RUN AS ADMINISTATOR'
  • You can check to make sure you can 'find' vpnc with this command:
$ which vpnc
/usr/sbin/vpnc
Repair the VPN Routing configuration script:
  • This is the bit that's broken in Win 7, or perhaps all x64 bit OS. The script in question is: /etc/vpnc/vpnc-script-win.js. I've posted my updated script here: Note that I added a bunch of debugging information to the script. You can make it much quieter by removing or commenting out the two echo commands inside the run function definition on lines 17 and 19.
  • The script supplied with vpnc relies on the execution of the "route print" command to extract the default gateway on this computer. For some reason the results of that command do not format the default gateway information in Win 7 x64 in the expected manner (perhaps other versions as well - I've seen references to this problem with Vista x64).
  • To fix the script, you need to replace the function getDefaultGateway with this one I hacked together:

function getDefaultGateway()
{
var output = run( "route print 0.0.0.0" ) ;
var pos = output.indexOf("0.0.0.0 0.0.0.0 ") + 30;
var gw = output.substring(pos,pos+15); // max length of ip address
gw = gw.substring(0,gw.indexOf(" ")); // trim at first space...
echo("Default Gateway: [" + gw + "]");
return gw;
}

  • I suck at regular expressions so I didn't try to use them for this. If you're better with regular expressions than I am feel free to post a comment with an improved function.
  • I also found it neccessary to add a smal pause to the script before adding the internal routes. Without the pause, the 'route add' commands were not adding correct routes in every case.

echo("Pausing for 4 seconds to allow the adapter to register itself correctly and therefore correct routing inferences made. You may need to supply a longer delay.");
WScript.Sleep(4000);

  • I put that sleep command just before line: if (env("CISCO_SPLIT_INC")) {
  • I also noticed that after disconnecting (ctrl-c), there were still stray routes in the route table. I'm not sure that it matters because the route to the main vpn gateway is properly removed, but I went ahead and added some code to remove all of the routes added by the script when disconnecting:

run("route delete " + env("VPNGATEWAY") + " mask 255.255.255.255");

//remove internal network routes
if (env("CISCO_SPLIT_INC")) {
for (var i = 0 ; i < parseInt(env("CISCO_SPLIT_INC")); i++) {
var network = env("CISCO_SPLIT_INC_" + i + "_ADDR");
run("route delete " + network );
}

  • Alternatively, I've posted an updated file here: Just copy that to your /etc/vpnc folder.
Create your VPNC Configuration file:
  • You need to convert your old Cisco .pcf file into a config file for vpnc in order to get the shared secret included in that file carried over correctly, and to make sure you have the ip address of the vpn gateway, etc. So, first copy your existing cisco .pcf file into somewhere convenient. /tmp is pretty convenient. Then run the pcf2vpnc utility:
$ cd /tmp
$ pcf2vpnc <old Cisco filename>.pcf /etc/vpnc/<profilename>.conf
  • <profilename> is whatever unique name you want it to be. Note that we're saving the new configuration file to /etc/vpnc. You could always copy the created file to that location later.
  • Now, edit the file it created with your favorite unix friendly editor. Windows Notepad is no good. Go download Notepad++ right now; I'll wait. Got it? OK. Open the file /etc/vpnc/<profilename>.conf
  • You'll need to add a few items to the file:
Interface name <NameOfTheInterfaceYouPickedEarlier> # mine is: CiscoVPN
Interface mode tap
Pidfile /var/run/vpnc/<uniqueName>.pid # need a unique one per profile, so may as well use <profilename>.pid
Local Port 0 #auto selects a port
NAT Traversal Mode force-natt
No Detach
  • You really shouldn't do it, but you can also enter your password in this config file if you don't mind that password being there in plain text. The syntax for that is:
Xauth password <yourpassword> # You've got a secure computer, right? Really? Are you sure?
  • If you have any trouble, you can add a debug flag to your config file:
Debug 1 # valid values: 1-3, 99. 99 = everything, including authorization information (passwords), so be careful

Putting it all together:
  • Actually there's not much to put together, you just need to use your shiny new vpnc command:
$ vpnc <profileNameWithoutExtension>
  • You could drop that command into an executable file to save a few keystrokes. I'll leave figuring out how to launch the cygwin bash shell and executing that command from a windows batch file (and so an icon on your desktop) up to you.


I hope this works for you, but I can't guarantee anything! Feel free to post comments with your experiences.

Update 10 Aug 09:

I have had issues with routing within the network if the vpn concentrator provides me with a new IP address. I'm not sure exactly why that happened or what to do in order to fix the routing - I'm not a network routing guru. I did figure out how to work around this issue though. In Control Panel\Network and Internet\Network Connections I went to the TCP/IPv4 settings and switched to dhcp ('obtain an IP address automatically, Obtain DNS server address automatically). These settings are overwritten when connecting to the vpn again, but for some reason making that change clears out something so that when connecting via vpnc the routing works again. For today anyways. :)

Cheers,

Allan

Update 11 Aug 09:
I forgot to mention that in my research, this VPN client: http://www.shrew.net/software was mentioned by a number of folks as a valid free alternative. It actually looks pretty good, but I didn't try it, as I wanted to get the vpnc option figured out. I always install cygwin on my windows boxes so I didn't mind going down this route. If anyone has experience with the shrewsoft client on x64 bit windows, please comment here.
]]> allan Speeding up the build – ditch the SSD and go for the RAM drive : Jeffrey Palermo (.com) 2009-07-01T06:41:00-08:00 2009-07-01T06:41:00-08:00 tag:ramblings,2012:AllansRamblings.39 Speeding up the build – ditch the SSD and go for the RAM drive : Jeffrey Palermo (.com)Thought I'd pass along props to Jeffrey Palermo for doing a great job on his post about using RAM drives to improve developer workstation performance - specifically running builds & unit tests.  He's done a lot of experimenting and has the data to back up his choices.  Definitely something to consider doing these days with RAM so cheap. Speeding up the build – ditch the SSD and go for the RAM drive : Jeffrey Palermo (.com)

Thought I'd pass along props to Jeffrey Palermo for doing a great job on his post about using RAM drives to improve developer workstation performance - specifically running builds & unit tests. He's done a lot of experimenting and has the data to back up his choices. Definitely something to consider doing these days with RAM so cheap.
]]> allan Kaban Development Oversimplified - putting limits on parts of your development cycle 2009-05-27T09:45:00-08:00 2009-05-27T09:45:00-08:00 tag:ramblings,2012:AllansRamblings.38 Jeff Patton has a great blog about Agile product development practices - mostly from the product/design perspective, but he's got some good project management information in there too.  He doesn't post very often, but his posts are real winners. His latest article is about Kanban Development - something that has been gaining traction and awareness in the Agile development community for the last year or two.  I won't expound much on it here, as Jeff does a much better job of it than I would - read his post! Kanban development comes from Toyota's famously efficient and high quality product development & manufacturing teams - Kanban means literally visual card (or board), and is about the use of simple visual devices to control the flow of work through any system.  A common problem in software development is having too many unfinished work items on a person or team's plate at any one time.  This is usually a symptom of people not completely finishing the items before moving on to the next one, because they've run into some type of roadblock with their current item (need a question answered, waiting for the next build, or whatever).  The problem with this is that it increases the amount of context switching people have to - do by orders of magnitude in some cases.    The costs of context switching for people working in highly intellectual jobs like ours is very often grossly underestimated, because we're so used to multi-tasking and often our team culture often promotes it - people with a lot on their plate are looked up to as busy and productive, when they are really just spinning their wheels with lots of activity but little real progress. A related problem is that this increases the 'cycle time' - the average amount of calendar time it takes for items to be completed, which increases the risk that items will become stale.  For example it is much more efficient for a developer to fix bugs on an item that they coded today or yesterday than even a few days later.  The use of Kanban concepts can help with these problems by putting simple rules or governors on your software development workflow to simply not allow too many work items in any one stage or spot.  For instance: you could have a rule to not allow more than x units of work in the Implementation (coding) state.  If a business analyst is finished with another requirement/story/use case, they're not allowed to send it to the development queue.  And if the size of work sitting in the design queue is already maxed out, they're not allowed to pick up another story and start working on that.  What should they do then?  Perhaps help the testers test some of the items in their queue.  Perhaps go talk with customers about the latest release, or do some usability/user experience testing.  Anything but drop more work on the developers.  The same would go for developers - if the test queue is 'full', they can't drop more coded features onto the test team.  They should go help the testers with their work items, help set up the performance testing lab, help troubleshoot the latest escalation from support.  They could even take a bit of time for professional development by doing some reading or experimenting.  Having people help with addressing the current bottlenecks for the team keeps the work flowing through the system at a more reliable rate, and reduces the amount of context switching going on because there simply won't be so many work items in play at any one time.  It also reduces the cycle time time for items in the system, which means that the work is fresher in people's mind so that things like bugfixing and any rework from feedback are done much more efficiently. The more subtle benefit that this brings to the team is to bring the 'whole team together'.  It's no longer "someone else's problem" if they're overburdened.  It encourages teamwork and collaboration, and puts the focus on 'finished' to the end of the system (ready for production), rather than just finishing my tasks. These rules could be put into place by simple declaration of policy (another email from the manager), or implementing some sort of rules in your task tracking system, but the use of very visual mechanisms like a story board can be very helpful in not only enforcing the rules, but making sense of them for the team - they can 'see' that the test team is overburdened, so the decision to go and help them out becomes a natural decision that doesn't require top-down direction.  Jeff's article above has some good examples of how story boards can be used for this.  For distributed teams, that story board may need to become a dashboard on your team's wiki, or you could experiment with tools like cardmeeting.com. There are a number of variations on this theme that can also be useful - minimum size in certain queues, and maximum sizes for any one person.  Remember when experimenting with things like this to always do a retrospective with your team at the end of the iteration to get feedback and make decisions about improvements. I know this is a pretty light treatment on the subject, but hopefully this has introduced the concept well enough that you're encouraged to do a bit more research on your own and dream about how this might help your team work together more efficiently. Cheers, Allan

Jeff Patton has a great blog about Agile product development practices - mostly from the product/design perspective, but he's got some good project management information in there too. He doesn't post very often, but his posts are real winners.

His latest article is about Kanban Development - something that has been gaining traction and awareness in the Agile development community for the last year or two. I won't expound much on it here, as Jeff does a much better job of it than I would - read his post!

Kanban development comes from Toyota's famously efficient and high quality product development & manufacturing teams - Kanban means literally visual card (or board), and is about the use of simple visual devices to control the flow of work through any system.

A common problem in software development is having too many unfinished work items on a person or team's plate at any one time. This is usually a symptom of people not completely finishing the items before moving on to the next one, because they've run into some type of roadblock with their current item (need a question answered, waiting for the next build, or whatever).

The problem with this is that it increases the amount of context switching people have to - do by orders of magnitude in some cases. The costs of context switching for people working in highly intellectual jobs like ours is very often grossly underestimated, because we're so used to multi-tasking and often our team culture often promotes it - people with a lot on their plate are looked up to as busy and productive, when they are really just spinning their wheels with lots of activity but little real progress.

A related problem is that this increases the 'cycle time' - the average amount of calendar time it takes for items to be completed, which increases the risk that items will become stale. For example it is much more efficient for a developer to fix bugs on an item that they coded today or yesterday than even a few days later.

The use of Kanban concepts can help with these problems by putting simple rules or governors on your software development workflow to simply not allow too many work items in any one stage or spot. For instance: you could have a rule to not allow more than x units of work in the Implementation (coding) state. If a business analyst is finished with another requirement/story/use case, they're not allowed to send it to the development queue. And if the size of work sitting in the design queue is already maxed out, they're not allowed to pick up another story and start working on that. What should they do then? Perhaps help the testers test some of the items in their queue. Perhaps go talk with customers about the latest release, or do some usability/user experience testing. Anything but drop more work on the developers. The same would go for developers - if the test queue is 'full', they can't drop more coded features onto the test team. They should go help the testers with their work items, help set up the performance testing lab, help troubleshoot the latest escalation from support. They could even take a bit of time for professional development by doing some reading or experimenting.

Having people help with addressing the current bottlenecks for the team keeps the work flowing through the system at a more reliable rate, and reduces the amount of context switching going on because there simply won't be so many work items in play at any one time. It also reduces the cycle time time for items in the system, which means that the work is fresher in people's mind so that things like bugfixing and any rework from feedback are done much more efficiently.

The more subtle benefit that this brings to the team is to bring the 'whole team together'. It's no longer "someone else's problem" if they're overburdened. It encourages teamwork and collaboration, and puts the focus on 'finished' to the end of the system (ready for production), rather than just finishing my tasks.

These rules could be put into place by simple declaration of policy (another email from the manager), or implementing some sort of rules in your task tracking system, but the use of very visual mechanisms like a story board can be very helpful in not only enforcing the rules, but making sense of them for the team - they can 'see' that the test team is overburdened, so the decision to go and help them out becomes a natural decision that doesn't require top-down direction. Jeff's article above has some good examples of how story boards can be used for this. For distributed teams, that story board may need to become a dashboard on your team's wiki, or you could experiment with tools like cardmeeting.com.

There are a number of variations on this theme that can also be useful - minimum size in certain queues, and maximum sizes for any one person. Remember when experimenting with things like this to always do a retrospective with your team at the end of the iteration to get feedback and make decisions about improvements.

I know this is a pretty light treatment on the subject, but hopefully this has introduced the concept well enough that you're encouraged to do a bit more research on your own and dream about how this might help your team work together more efficiently.

Cheers,

Allan

]]> allan Quality & Broken Windows 2009-05-26T08:40:00-08:00 2009-05-26T08:38:00-08:00 tag:ramblings,2012:AllansRamblings.37 The authors of the Pragmatic Programmer (great book btw, highly recommended) have a lot of insights into the craft of software development, but in my opinion one of the most powerful ideas is that &quot;living with broken windows&quot; drives quality downward in a depressing spiral of entropy.  Well, they put it a lot more eloquently than that, but the idea is that when you're working in a codebase that's littered with problems, bugs, sloppiness, etc, you and your team are much more likely to do sloppy work.  Your standards are lowered by the sheer weight of the code around you, in spite of your best intentions and even explicit attempts to do high-quality work.  This is a great argument for continued refactoring of existing code, even if that code is more or less functional.  It can affect the quality of new code just by its presence in the same codebase.  It's like a poison, or at least a bad smelling swamp gas... I won't attempt to elaborate much more on the concept - this article explains it much better than I'm likely to.  Go ahead and use up some of your daily or weekly geek reading time on that article - it's time well spent. What's your experience like?  Have trouble keeping high standards when working on crappy legacy code that someone-who's-no-longer-working-here-for-a-good-reason wrote?  Wish you could rewrite that whole module or function from scratch but just don't think you have the time?  Can't convince your Dev or Product manager to let you have the time to refactor that painful and buggy sub-system?  Or, have you been able to clean up parts of your codebase with positive results? Was it worth it? Do you think it really makes much of a difference?  Or, can you manage to keep your work up to all the awesome standards that you have in spite of the muck that you're wading around in? What do YOU think? P.S. Don't take this as suggesting that all of the legacy code is crap - it's not.  There's a lot of really great code out there, I know.  And you don't need to look only at 'legacy' code to find sloppiness either.  These are big generalizations, but I think they're useful ones that apply to much more than just code.

The authors of the Pragmatic Programmer (great book btw, highly recommended) have a lot of insights into the craft of software development, but in my opinion one of the most powerful ideas is that "living with broken windows" drives quality downward in a depressing spiral of entropy. Well, they put it a lot more eloquently than that, but the idea is that when you're working in a codebase that's littered with problems, bugs, sloppiness, etc, you and your team are much more likely to do sloppy work. Your standards are lowered by the sheer weight of the code around you, in spite of your best intentions and even explicit attempts to do high-quality work. This is a great argument for continued refactoring of existing code, even if that code is more or less functional. It can affect the quality of new code just by its presence in the same codebase. It's like a poison, or at least a bad smelling swamp gas...

I won't attempt to elaborate much more on the concept - this article explains it much better than I'm likely to. Go ahead and use up some of your daily or weekly geek reading time on that article - it's time well spent.

What's your experience like? Have trouble keeping high standards when working on crappy legacy code that someone-who's-no-longer-working-here-for-a-good-reason wrote? Wish you could rewrite that whole module or function from scratch but just don't think you have the time? Can't convince your Dev or Product manager to let you have the time to refactor that painful and buggy sub-system? Or, have you been able to clean up parts of your codebase with positive results? Was it worth it? Do you think it really makes much of a difference? Or, can you manage to keep your work up to all the awesome standards that you have in spite of the muck that you're wading around in?

What do YOU think?

P.S. Don't take this as suggesting that all of the legacy code is crap - it's not. There's a lot of really great code out there, I know. And you don't need to look only at 'legacy' code to find sloppiness either. These are big generalizations, but I think they're useful ones that apply to much more than just code.

]]> allan ELMAH - an Amazing asp.net error logging monitoring tool 2009-08-16T14:45:00-08:00 2009-04-24T14:35:00-08:00 tag:ramblings,2012:AllansRamblings.36 I saw this referenced on Jeff Atwood's blog a little while ago and meant to pass it along to a few folks, and now I see that Scott Hanselman has just blogged about it as well. Scott's article is probably the best place to start. for more detailed information (I guess other than the project's site) I saw this referenced on Jeff Atwood's blog a little while ago and meant to pass it along to a few folks, and now I see that Scott Hanselman has just blogged about it as well. Scott's article is probably the best place to start. for more detailed information (I guess other than the project's site)

Here's the blurb from ELMAH'S site:

ELMAH (Error Logging Modules and Handlers) is an application-wide error logging facility that is completely pluggable. It can be dynamically added to a running ASP.NET web application, or even all ASP.NET web applications on a machine, without any need for re-compilation or re-deployment.
Once ELMAH has been dropped into a running web application and configured appropriately, you get the following facilites without changing a single line of your code
    • Logging of nearly all unhandled exceptions.
    • A web page to remotely view the entire log of recoded exceptions.
    • A web page to remotely view the full details of any one logged exception.
    • In many cases, you can review the original yellow screen of death that ASP.NET generated for a given exception, even with customErrors mode turned off.
    • An e-mail notification of each error at the time it occurs.
    • An RSS feed of the last 15 errors from the log.
    • A number of backing storage implementations for the log, including in-memory, Microsoft SQL Server and several contributed by the community.

I don't know about you, but this is very exciting for me - exactly the type of thing that I've been encouraging folks to somehow get onto their development schedules. Things like this help drive quality to the next level by not only giving you tools to diagnose problems better, they increase the visibility of problems by orders of magnitude. No longer are errors hidden away in some error log on the server which isn't easily accessed by the average dev. RSS feeds of the latest error reports to drop on your iGoogle homepage! That's the bomb!

As with anything like this, be careful what you ask for, because you could easily be innundated with error reports to the point where you may have a hard time absorbing the information. But its a SUPER motivator to fix those seemingly benign issues that wind up generating the bulk of your exceptions...

I'd love to hear about anyone's experience with this or similar tools, or if it inspires you to brew up something similar for your other (non .net) projects.

]]> allan About Me 2011-04-28T19:28:00-08:00 2008-03-23T14:36:00-08:00 tag:ramblings,2012:AllansRamblings.12 Hi, I'm Allan Nienhuis.  At work I help software development managers keep their teams productive and happy at the Active Network (my official title is 'Sr Director, Software Development').  I also help my team in Xian and Chengdu, China grow at an incredible pace (over 400 people at this point).  I'm still amazed that my wife Shelley actually married me (28th anniversary this year!).  Both of our children (daughter Danielle, son Noel) have moved out of the house kind of recently so we're experiencing the 'empty nest' thing first hand.  We're now living on the beatiful waterfront of Pender Harbour, which is a bit north of Vancouver, BC in an area called 'The Sunshine Coast', although we lived in Abbotsford, BC for over 20 years.  You can find some pictures of our place here. Our Christian faith is imporant to us, and for many years we've supported and encouraged others to support several hands-on ministries - www.onemileclinic.org, www.partnersforhaiti.org, and the work of Norm and Jen Weir serving people in remote communities in BC.  I suppose my first post on why I bothered putting up this blog in the first place will probably get kicked off the front page after a while, so I'll put a link to it here. Hi, I'm Allan Nienhuis. At work I help software development managers keep their teams productive and happy at the Active Network (my official title is 'Sr Director, Software Development'). I also help my team in Xian and Chengdu, China grow at an incredible pace (over 400 people at this point).

I'm still amazed that my wife Shelley actually married me (28th anniversary this year!). Both of our children (daughter Danielle, son Noel) have moved out of the house kind of recently so we're experiencing the 'empty nest' thing first hand. We're now living on the beatiful waterfront of Pender Harbour, which is a bit north of Vancouver, BC in an area called 'The Sunshine Coast', although we lived in Abbotsford, BC for over 20 years. You can find some pictures of our place here. Our Christian faith is imporant to us, and for many years we've supported and encouraged others to support several hands-on ministries - www.onemileclinic.org, www.partnersforhaiti.org, and the work of Norm and Jen Weir serving people in remote communities in BC.

I suppose my first post on why I bothered putting up this blog in the first place will probably get kicked off the front page after a while, so I'll put a link to it here.

]]> allan