My developers toolbelt 2016

I cought a tweet a few days ago asking for your developers toolbelt, specifically on windows. And I gave a very short answer and mentioned I would blog about this:

So, this is a more elaborate answer to the developers toolbelt question. My default windows developer installation contains the following:

  • Windows 10 Professional (fully updated)
    • .NET Frameworks active
    • IIS Installed
    • Dev-Mode enabled
    • Linux Subsystem installed
    • Windows Defender as Antivirus solution & default Firewall
      (no external security software)
  • Dropbox
  • 1Password
  • IE, Edge, Chrome, Firefox (for testing, yes, I do quite a bit web dev 🙂 )
  • Git for Windows for commandline usage
  • SourceTree as my graphical git client*
  • Beyond Compare as my diff tool
  • Cmder as my console of choice
    (my previous blog post is about using the Linux bash on Windows in Cmder)
  • Node Version Manager nvm for Windows, and as such a lot of node versions
  • Primary IDE: Visual Studio 2015 (with ReSharper Ultimate)
  • Secondary IDE: WebStorm
  • .NET Sourcepad: LinqPad
  • Primary Database: SQL Server 2016 Express
    other DBMS as required by projects.
  • dbForge Schema compare and dbForge Data compare
  • Office 365
    • OneNote for collaboration
    • PowerPoint for presentations
    • Outlook for e-mail comms
    • Word for occasional paperwork
  • Slack and TeamViewer for other comms / collab
  • VSCode as my main text editor
  • Atom as secondary text editor (i.e. large markdown files, where VSCode crashes)
  • Android Studio
  • Genymotion android emulator (quite a bit faster than the normal one)

In the list above, except for ReSharper, I am not listing additional addons / extensions to the other listed tools.

* – I also tried GitKraken, Tower for Windows and the GitHub client, but they are – in my opinion – not as usable as SourceTree. Especially Tower wastes too much screen estate.

Ask a Ninja: Current state of static site generators

Over the course of the last weekend I tried to built a website for a side project of mine (gaming related). To broaden my horizon, and to be able to host the website cheap and fast, I wanted to use a static site generator for it.

First try: Jekyll. TL;DR: Does not work on windows.

Since Jekyll is directly supported by Github pages, and I wanted to host here, and a lot of other guys around on my Twitter timeline use Jekyll, I thought this is the way to go.

Jekyll does not state that it would work under Windows. It is also not officially supported there. So this is not a great surprise. There were, however, some tutorials / manuals around that described how it should be able to run on Windows, which I wanted to follow. I installed ruby, and right from the the first try to install the Jekyll gem failed hard. So I could not even follow those instructions, as already the installation failed because of some strange SSL errors.

That’s when I looked for an alternative. I found the page Staticgen.com that lists a couple of projects related to this topic. This lead me to the next step.

Second try: Pretzel. TL;DR: Not stable. Crashes all the time.

As I am a developer that feels himself comfortable on the .NET platform, I looked for something compatible with Jekyll that, mabye, would run on Windows and on Mono on my Mac. Pretzel was the candidate. I installed Pretzel, and it worked with the template that ships with it.

Sadly, as soon as I started to throw in my own HTML, CSS and JS, the Pretzel preview server crashed at least at every second request. So it was changing files, restart Pretzel, preview. Change files, restart Pretzel, preview, and so on. I did this like half an hour when I became too annoyed to follow this path further.

At the end of the day, I want to build a website, and not debug a tool that will generate some HTML for me. So it was time to look for some alternative again. Since there weren’t any other alternatives on .NET that were Jekyll compatible, I thought about Node as a platform. I know that Node JS runs on Windows and my Mac, I had some previous experience with Node and Javascript is not that bad.

I did not go to StaticGen this time, because a handful of people on my timeline also started to use a Node based static site generator: Wintersmith. So without double checking the facts and blindly trusting my tech-colleagues, I downloaded it.

Third try: Wintersmith. TL;DR: Plugins don’t work and/or are too complicated. Not enough docs.

To be honest, I was totally turned off by the default template engine Jade. I have a full HTML template, and Jade is nowhere near HTML. Some people may find that cool, but it does not represent the way I want to work with my website.

Gladly, the template system can be changed, and I switched to Handlebars, as I liked the easy and lightweight mixin with HTML. Then the difficulties began.

The HTML template I was going to use requires Bootstrap (which requires jQuery), FontAwesome, I used two additional webfonts to match a certain website style, and alltogether I ended up with a few javascript and css files. I wanted them combined and minified, but the most promising Wintersmith plugin kelvin was not ported to Wintersmith 2.

Another plugin problem was wintersmith-tags. First of all, the plugin exposes methods to be called in the template. Something Handlebars is not capable of. Luckily the switch to Nunjucks for the templating engine was more or less done quickly without a lot of effort, but then I noticed that wintersmith-tags would not, despite the documentation stating otherwise, list all available tags on call of the corresponding method. I just got back an empty array.

That the plugins are mostly written in Coffescript, which I am not used to read, does not make it any better. The fact that wintersmith-tags was the only plugin with a rough documentation makes it even more difficult to work with. Up to the point where I gave up I had 11 plugins installed, none working as I intended them to.

Conclusion:

Fact is: I’m totally frustrated about the extremely poor experience I had over the last weekend.

I will give it a last shot with Hexo. It seems more active than Wintersmith, is pure Javascript (no Coffeescript, so for me better to read and understand), and it seems to have a larger installation base, so the chances are higher to get questions answered by the community. But if that does not work out too, I already came up with my plan B:

Let go of all these static site gens available out there. I’m probably going another route: I am a developer with deep knowledge in .NET. And .NET, especially ASP.NET MVC, Razor, Web.Optimization etc. has all the stuff required to built a great website, right at my fingertips. But I still want simple HTML delivered.

It is likely I’m going to grab ASP.NET vNext and built my own static site gen on top of it. Using the next .NET version it will also run cross-platform and I can use it on my Mac too.

This way I can make sure it works as I want it to, and since the guys using Wintersmith (and by the way are not really happy with it too) are also mainly former .NET devs, I probably have a good chance to get the first users except me, too. But I really hope Hexo is going to work out. Doing that much work just to get a simple but modern website on, is quite ridiculous. A decade ago it was a lot easier, but expecations have risen since then. A lot.

Update 2015-02-10: I will postpone my exploration of Hexo a little bit. It seems the project is still alive and just a few days ago release it’s version 3 RC. I can’t figure out if Hexo does support Nunjucks but I’m getting in touch with its creator and will find out.

Custom deployment scripts – with mstest – for Windows Azure Website git deployment

I just started another project. It is hosted on Windows Azure and I’m using Git deployment for this website.

This was very fine and I am extremely impressed how easy it was to get started with it. Then I ran into a little problem.

Sidenote: My project relies on NuGet packages, and I, personally, have the strong opinion that compiled stuff does not belong into my source code versioning system. This is why I did not check in the NuGet.exe into Git, but just the NuGet.config and NuGet.Targets files configured to download NuGet.exe when it’s missing. Of course I make my build dependent on a NuGet package server, but since I could host my own gallery on a custom domain, and configure that domain in my NuGet.config, I could take control over this dependency at any time.

I wanted my project to incorporate information about the Git commit hash it is built from, the Git branch it was built from and other little details. For that the MsBuild Community Tasks project offers some nice helpers. So I added the NuGet package of this project to my solution.

The Problems

Now there is this chicken-egg problem: When MsBuild encounters a UsingTask declaration, it automatically loads the assembly that contains the task. If that assembly is not there, using the task will fail. Now, the NuGet download of the packages – including the task library – happens as part of the build. That is, after the project files are loaded. So the fresh downloaded file was not found when importing the projects and… the build fails.

To avoid this problem, I cheated a little bit on MsBuild: I added another project to my solution that also has the MsBuild Community Task project listed in it’s packaged.config. Then I manually set my web application project to be build after this ‘BuildSupport’ project. Now the BuildSupport project build downloads the community task library, which is then available when the project import is defined in the web application’s project file. It’s just a small cheat, though.

Then the next problem: The BuildSupport project is not actually ‘required’ to build the website project, and so the Git deployment build process simply does not build it. The task library is not downloaded prior to executing the actual build process of the application, and so it fails. I could not get the project to build the ‘BuildSupport’ project before the actual web application on Azure.

The Solution

After a little bit of research I found this can be achieved by using a custom deployment script.
I was a bit afraid that I had to figure out how the actual deployment works to add a step just in front of the actual compile, but there is some infrastructure in place to help us out with that.

For a .NET developer this will feel strange, but you’ll need node.js in the first place. The Windows Azure Command Line Tools are a node.js package, and we’ll need that to get started with the actual live deployment script. So, after installing node.js, we’re going to install the package:

npm install azure-cli -g

This will globally install the Azure CLI for use on our console. Now we navigate to our solution directory and let the Azure CLI generate the deployment script that will automatically run to deploy our application to Azure if we don’t do anything custom:

azure site deploymentscript --aspWAP ApplicationFolderApplication.csproj -s Solution.sln

This will generate two files for you. First there is a .deployment file. This is a file structured like a oldfashioned .ini configuration file, telling Azure that there is a custom deployment file and how its name is. It’s content simply is:

[config]
command = deploy.cmd

It also reveals the second generated file, the actual deployment script called deploy.cmd. This is the interesting part for us so far. I’m not posting the full script but rather go through the sections.

First there is a check that node.js is available. It is assumed that this is available on Azure, but to test the deployment script locally you’ll also need node.js. We just installed it, so we’re all set, but the next one checking out the solution could be missing node.

Then the script defines some environment variables for folders. Like where the build artifacts will be placed and where the actual files to deploy will be placed. This defaults to /artifacts/wwwroot and can be overridden by setting the corresponding environment variables before the deployment.

In a thirds step, the script checks if kudu is installed. Kudu is the actual deployment engine running on Azure, and is also capable of running on your machine. After that additional paths are configured.

In the fourth step the actual compiling and deployment work is done, and the fifth is just some error handling.

So, let’s have a look at the actual important stuff in the file:

:: 1. Build to the temporary path
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%MyApplication.WebMyApplication.Web.csproj" /nologo /verbosity:m /t:Build /t:pipelinePreDeployCopyAllFilesToOneFolder /p:_PackageTempDir="%DEPLOYMENT_TEMP%";AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release /p:SolutionDir="%DEPLOYMENT_SOURCE%.\" %SCM_BUILD_ARGS%
IF !ERRORLEVEL! NEQ 0 goto error

:: 2. KuduSync
call %KUDU_SYNC_CMD% -v 50 -f "%DEPLOYMENT_TEMP%" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"
IF !ERRORLEVEL! NEQ 0 goto error

Now, thats actualy very easy: MsBuild is called for the web application project, and then Kudu is launched to do the actual deployment.

What we want to achieve now is to build the full solution upfront to have all required NuGet packages downloaded before the actual project is being built. And while we are actually getting our hands dirty in a custom deployment script, why don’t add running the unit tests of the project as part of the deployment? So, if a test fails, deployment will fail too. I think that’s a good idea.

So what I did was adding these two steps just in front of the two default steps:

:: 1. Build solution
echo Build solution
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%MySolution.sln" /nologo /verbosity:m /t:Build /p:_PackageTempDir="%DEPLOYMENT_TEMP%";AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release /p:SolutionDir="%DEPLOYMENT_SOURCE%.\" %SCM_BUILD_ARGS%
IF !ERRORLEVEL! NEQ 0 goto error

:: 2. Running tests
echo Running tests
vstest.console.exe "%DEPLOYMENT_SOURCE%MyApplication.Web.TestsbinReleaseMyApplication.Web.Tests.dll"
IF !ERRORLEVEL! NEQ 0 goto error

That’s it. I just copied the build line and pointed it to my solution, and I added a call to the MsTest tooling to run my tests.

So, with very little tweaking I could remove all dependencies to actual binaries I would have to check in otherwise and I have the Azure git deployment run my unit tests on every deployment. That’s what I call easy and convenient.

Setting up my infrastructure – Part 8: A little bit more evaluation

After a little break I’m back again with the next findings in my little pet project.

First of all I wanted to check out the build servers before diving into the bug trackers. So I set up the first assembly for my project, with just one class and a unit test for it. As I already mentioned in my requirements it is a .NET project and I want to have at least the very basics (building, unit testing etc.) covered.

My test case for this was very easy:
Set up a project in the build server, have it check out the sources of the project and let it build.
After building is okay, add the configuration for unit tests and let the test run.
Check if it builds when checking in new code, and check how it reacts when either the build breaks or unit tests fail.

So, now this is, what I encountered with the systems:

Bamboo

Without digging too deep into the documentation (Documentation? Yes, this is the url. Check.), I set up a build plan in Bamboo to build my project. I set up a first stage that should check out the project from source control and build it.

Well, the setup was done quickly, but the build would not start. What I did was the following: I configured two local agents, both are capable of running the build. The build plan had a single stage “Build” with two jobs: Checkout and call msbuild with my solution file.

When I manually selected ‘Run’ it was queued and ‘waiting to be build’ forever. When clicking onto it, and selecting the only stage with the hourglass besides it from the sidebar, it told me this status: “Status: Job has not yet been queued. Waiting for prior stages to complete.”.

This was where I went like WTF? What prior stages? This was the only existing stage in the only existing plan.
After spending several hours with this situation I decided to create a User account on my Bamboo evaluation installation for someone to support me and opened a support ticket at Atlassian.

A few hours later the support logged in – and just saw a failed build that started like 20 minutes before.
After that the server behaved like normal: A new checkin resulted in an almost immediate new build like I would have expected.

So, why did the build fail? I used a Git repository on BitBucket for the tests, and configured the version control settings to check out from Git. I though that was the obvious way to do it. I was wrong, as Bamboo would not check out the sources. As I later found out, since the Git (hint! hint!) repo is hosted on BitBucket, I needed to select ‘BitBucket’ instead of Git from the repository type selection to be able to use it. My dear. After that it worked.

So, after this little problem I went on to configure the unit tests. I added a new stage for the tests to the build plan and configured a new job that would call MsTest and run the tests.

That job failed on every approach to run it. It always told me that the assembly was not found.
To make a long (several hours over a few days!) story short: Obviously the different stages in a build plan are executed on the same agent, but in different working directories.

At the time of this evaluation there wasn’t anything hinting to that in the Atlassian docs. By now, there is a small article explaining that you need to configure Artifacts to bring build results from one stage into another, but without further explaining how to set up the artifacts in detail.

Back then, I tried to figure out the different directories but that required a lot of changes to my build scripts that wouldn’t work on the dev machines after that, and so I put the MsTest job in the build stage to have it execute.

Guess what? After changing the build stage it wouldn’t start again with the same delayed until forever problem I already had until I waited several hours when it suddenly executed.

After all, my experience with Bamboo wasn’t really turning me on.

TeamCity

I already mentioned that we use TeamCity at my work place. So setting up the initial project was a bit new to me, but I knew what the settings were and where I had to tune a little bit. So the initial setup of my project with two build steps build and test was done in a few minutes.

Of course everything worked from the very instane I clicked on run.
Then I went on to activate code coverage reports for the test build step by simply activating dotCover from a little drop down in the test build step. After the next build I had a complete test coverage report on TeamCity.

Conclusion

I struggled a lot with Bamboo. It has a steep learning curve and the documentation is.. well, let’s simply call it it could be better. A lot. The Atlassian tools in general, and Bamboo is no exception, are obviously powerful, but you have to spend a LOT of time with the system to get it running like you expect it to – and I don’t want to spend too much time fiddling around with my toolset.

TeamCity on the other hand is streamlined and guides you through the process of configuring your builds. Everything I needed was set up in a matter of minutes and everything worked directly as expected from the very beginning.

Actually, at this point it was clear for me to use the JetBrains tools and not Atlassians, but I still had a quick look at Jira and YouTrack, which I want to describe in a dedicated blog post.

See the other parts in this series:

[contentblock id=infrastructurelinks]

Setting up my infrastructure – Part 1: Basic tools

For my new pet project I want to use good and efficient tooling. Since I want to create a tool for me and other .NET developers and I feel at home on this platform, I’m going to use C# for the project.

I have my personal MSDN Professional subscription, and so I use Visual Studio 2012 Professional for development. I add my personal ReSharper licence for productivity and I chose Windows 8 Professional as my development OS (in a VM on my MacBook Air). Being totally in the Microsoft .NET ecosystem I’m also going to use MSBuild and MSTest.

Update: Talking about VM on my Mac, I use VMWare Fusion for that. I also have VMware Workstation running on my home server for my build server virtualization, but that will be part of another post.

For source code versionioning I chose Git. Mainly, because I feel that even if Mercurial currently has better tooling support on Windows, Git is more mainstream and tooling is becoming better. As Git clients I currently use the GitHub client and of course the official Git commandline client. I host my sources on BitBucket from Atlassian. They give you private repositories for free, and since I invited some guys I also can collaborate with 3 others if I want without the need to pay for a private shared repository.

Besides that, I of course have the usual .NET developer tools like The Regulator for working with regular expressions, LinqPad for small test thingies and DotPeek as my decompiler.

Now, besides that I need additional tooling to keep track of my tasks, so I need a bug / issue / task tracker. And I don’t want to build releases manually or do manual testing, so I will need some sort of automatic build & test tooling, which leads me to some continuous integration / build server. Chosing which tool is best here will take some time, and so I started to evaluate different solutions. More on that in a separate post.

So the toolset for my pet project is right now:

Continue with the next part, or see the other parts in this series:

[contentblock id=infrastructurelinks]