2018-01 on Funky Si's Test

Flexible architecture and interfaces

funkysi1701@gmail.com (funkysi1701) — Mon, 29 Jan 2018 00:00:00 +0000

I have blogged a few times about interfaces , and how useful they are for producing good quality maintainable code. Let’s look at a problem and the solution I came up with which I am quite proud of.

As previously mentioned I am in the process of moving images from AWS to Azure blob storage. Now that the actual files themselves have been moved I need to change the code that references them.

Now I could find all the code that uses the AWS API and replace it with the Azure API but I am not very good at predicting the future, we may stay on Azure for a while, we may move to AWS or Google Cloud, or we may want to go back to files sitting on a server.

Lets try and code a solution that is as flexible as possible. As you have probably guessed I am going to create an interface.

At first I thought about creating an interface called ICloudStorage , however this isn’t flexible enough as what happens if we go back to sticking files on a server so instead I created IStorage.

I created three classes that implemented IStorage, AWSStorage , AzureStorage and mostly for testing at the moment FileStorage. I then created a class Storage that would call these three classes. Initially I created it like this

public class Storage
{
 private IStorage _repo;
 public Storage(IStorage repo)
 {
 _repo = repo;
 }
}

However this would require I call it like Storage(new AzureStorage()) and I would need to know everywhere in my code which implementation I want to use. This isn’t too bad as when we change from AWS to Azure we would need to do a find and replace throughout the code and replace all AWSStorage and make them AzureStorage.

However we can do better than that.

public class Storage
{
 private IStorage _repo;
 public Storage()
 {
 Type obj = Type.GetType(ConfigurationManager.AppSettings["DefaultStorageRepository"]);
 ConstructorInfo constructor = obj.GetConstructor(new Type[] { });
 _repo = (IStorage)constructor.Invoke(null);
 }
}

This code will read from the web.config which implementation to use and that will decide which class to call. This means that to change from AWS to Azure we do not need to redeploy any code, all we need to do is change the web.config.

Let’s look at the three lines and see if we can understand what is happening.

Type.GetType() looks straight forward and gets the type from the web.config

obj.GetConstructor() This gets the constructor for the type we have just found.

constructor.Invoke This then invokes the constructor and it then gets cast to the interface so can be used by the _repo variable.

This is all fairly simple and makes sense, however it has produced some very flexible code and allows the code to be extended without recompiling.

Let’s look at a hypothetical example. We want to add support for Google Cloud Storage. All we need to do is create a class library which implements the IStorage interface, place the compiled binary in the website and update the web.config to reference it. I haven’t tried this hypothetical example so it might be more complex than I think but in theory it should work.

I am pretty excited at how flexible this code can be, hopefully I will use code like this more often now I understand it.

Moving files into blob storage

funkysi1701@gmail.com (funkysi1701) — Mon, 22 Jan 2018 00:00:00 +0000

We are in the process of moving our companies websites onto the Azure platform. One of the challenges was to move image files out of the website project into blob storage. This week I have moved 150,000 of them.

One thing I keep banging on about is that your source code should not contain data. If it does every time you do a deployment you need to consider where these images are located and ensure you don’t overwrite or loose any. It also goes without saying that deployments of a few Mb are a lot quicker than deployments of 100s of Mb.

Azure blob storage also gives you advantages like distributing storage across multiple datacenters which would be impossible with traditional files on a server.

So now that we have established that this is a good idea lets look at how we could move large amounts of data. In my case all the filenames are stored in a SQL database so the plan of action was to simply loop through the files in the database, download from current storage (either locally or other cloud storage), upload to Azure and tidy up afterwards. Due to the number of images I am going to update the database and mark when a file has been processed so I can do the move over several days.

This is my code

var source = "https://example.com/images/";
var tmp = Server.MapPath("~/tmp/");
if (!Directory.Exists(tmp))
{
 Directory.CreateDirectory(tmp);
}
var fixturePhotos = db.Images.Where(x => x.Moved == null || x.Moved == 0).Take(id);

foreach (var photo in fixturePhotos)
{
 try
 {
 string path = getFilePath(photo.FileName);

 if (!Directory.Exists(Server.MapPath("~/tmp/" + path)))
 {
 Directory.CreateDirectory(Server.MapPath("~/tmp/" + path));
 }
 WebClient WebClient = new WebClient();
 WebClient.DownloadFile(source + photo.FileName, tmp + photo.FileName);
 FileUploader f = new FileUploader(tmp + photo.FileName, photo.FileName);
 System.IO.File.Delete(tmp + photo.FileName);
 photo.Moved = 1;
 }
 catch
 {
 photo.Moved = 0;
 }
}
db.SaveChanges();
if (Directory.Exists(tmp))
{
 Directory.Delete(tmp, true);
}

First of all I create a tmp folder in the root of my website if it doesn’t exist to store my images temporarily.

I then use an entity framework model to query the database that haven’t been moved, and I use the take() method to limit how many results I process. (I have been passing in 1000 at a time)

I then use a foreach loop over all these files to perform the following actions.

Create additional subfolders if the filename variable stored in the database isn’t actually a filename but a filepath, note you will have to split filename and filepath which I haven’t included code for here.
Download file from the original url and save into the temporary folder
Upload to Azure
Delete temporary file
Update database giving a success or fail

Once the foreach is finished I commit the database changes and delete the temporary folder. I am sure there must be other ways to do this transfer but this was quick and easy to setup and now I have a copy of all the files in Azure storage so I can test out other issues with my website.

One last tip about how to schedule this code. I called the above code from a MVC controller and then wrote a Azure Function to call this code on a schedule.

Website UI Testing

funkysi1701@gmail.com (funkysi1701) — Mon, 15 Jan 2018 00:00:00 +0000

Last week I looked at testing the UI of mobile apps , this week lets look at how we could do a similar thing for websites.

Testing the user interface is not an excuse for a lack of unit tests . Testing the user interface takes longer so for keep creating your small unit tests that can be run after ever build. That said lets look at how you create a UI test.

Create a Unit Test project as normal. Now install the following nuget packages

Selenium.WebDriver.ChromeDriverSelenium.WebDriverSelenium.WebDriver.PhantomJS.Xplatform

I am going to be using Selenium to achieve my website testing and I am going to concentrate on the Chrome browser. However packages exist for other browsers so have a look at the following and I expect there are others as well.

Selenium.WebDriver.IEDriverSelenium.Firefox.WebDriver

Selenium started life as a plugin for Firefox to help create automated tests, however the latest version of Firefox is not compatible with the plugin as I write this. I have not had to install any plugins or extensions to my browsers to achieve my testing.

Enough talk lets write a test. First we create two instance variables to store the baseURL and the driver for the browser you are using.

private string baseURL = "https://www.example.com/";
private RemoteWebDriver driver;

Next we need to set things up for the test to run. This creates an instance of the chrome driver, maximizes the window and sets it to wait 30 seconds before timing out.

[TestInitialize()]
public void MyTestInitialize()
{
 driver = new ChromeDriver();
 driver.Manage().Window.Maximize();
 driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(30));
}

Now comes the actual test. We navigate to a URL and then compare the title of the page loaded with a know value with a Assert statement like you would find in a unit test.

[TestMethod]
public void CheckBrowserTitle()
{
 driver.Navigate().GoToUrl(this.baseURL);
 Assert.AreEqual("Home Page", driver.Title);
}

Finally we need to tidy up after ourselves.

[TestCleanup()]
public void MyTestCleanup()
{
 driver.Quit();
}

If you are used to writing tests you will know that the are usually constructed in three sections Arrange, Act and Assert. The Arrange is done in the initialize method, which makes the actual test much simpler, the first line does the Act and the last line does the Assert.

Now we have written a simple test lets look at something more complex.

driver.FindElementByLinkTest("click")

This finds any Link on the page which is click. Be careful as the string need to be exactly what appears on screen it may well be easier to specify by id or class or something that doesn’t change as often. By adding .click() on the end of this command Selenium will click on the link and you can navigate to a new page.

What about submitting form data? Well you can find the element you want to fill in and add .SendKeys(“example text”) or .Submit() and this will fill in and submit form data.

What about a screenshot?

Screenshot ss = driver.GetScreenshot();
ss.SaveAsFile("test.jpg",System.Drawing.Imaging.ImageFormat.Jpeg);

I have only just started playing around with web UI tests but you can see there is a fair bit you can do.

Mobile App UI Testing

funkysi1701@gmail.com (funkysi1701) — Mon, 08 Jan 2018 00:00:00 +0000

Since I started creating an android app I have been writing simple UI tests.

I have been taking advantage of the Visual Studio App Center which allows you to test against hundreds of different devices in the Test Cloud.

In order to write a UI test create a UI Test App, this makes use of the nuget package Xamarin UI Test. By default you will now have a test called AppLaunches which will take a screenshot of you app after it starts.

You can now run this test against any device from Visual Studio assuming you have it physically plugged into your machine. However, how do you run against the Test Cloud?

To run against the Test Cloud you need to first install node.js. Now you can install the appcenter cli with the following command.

**npm** install -g appcenter-cli

To run the tests in the Test Cloud run the following command

**appcenter** test run uitest --app "<username>/<appname>" --devices "<username>/<deviceset>" --app-path _pathToFile.apk_ --test-series "master" --locale "en_US" --build-dir _pathToUITestBuildDir_

where is your appcenter username, is the name of your app in appcenter and is the group of apps you have created in app center to test against.

Look at device sets in the test section of the appcenter and click the new device set button. You can then search for any device you like and add it to a set, as I write this there are 244 devices you can test against.

It is currently not possible within app center to run tests against a new build, however if you build your app in VSTS as well you can create build or release step that runs it. As the Visual Studio app center is still under development I wouldn’t be surprised if it is added at some point.

In VSTS look for Mobile Center Test in the definitions and you can specify the same variables as specified in the command line above.

Now how do you actually write a useful UI test? I mentioned above you get a default test which contains the following code, this takes a screenshot of your app. However you don’t have to include this when using the Test Cloud as screenshots are included for free.

app.Screenshot("First screen.");

Lets look at what else is included in app

app.Repl();

This starts an interactive REPL (Read-Eval-Print-Loop) which lets you explore what is on screen in your app and pauses execution. I don’t include this in my tests, however I do make use of it to explore what is on screen and what tests I might make use of.

app.Tap(c => c.Marked("Button"));

This taps an element on screen called Button. There is also a method called TapCoordinates which would allow you to click anywhere you like.

app.WaitForElement(c => c.Marked("View"));

After clicking a button you are probably going to want to wait for the app to load extra data or a new screen. The WaitForElement waits for an element to appear on screen. There are also methods that wait a period of time or wait until an element no longer exists.

These are the main methods I have used so far, however there is an extensive list including methods for scrolling, swiping, pinching and adjusting the volume buttons. So you should be able to test all manner of app functionality and if you make use of the Test Clouds will know which devices are causing problems.

Lets see what 2018 can do!

funkysi1701@gmail.com (funkysi1701) — Mon, 01 Jan 2018 00:00:00 +0000

It is 2018 so it must be time to think about what my plans and goals are for the new year.

Buy a house

This a huge goal for me. For many years I was content with renting and thought owning my own home wasn’t for me, but then I went and started a family and my career started going in the right direction and my thoughts about this have changed.

It is going to take a lot of work to make this goal happen in 2018. First I need to save enough money for the deposit, which is far from easy with a wife and two children. Second I need to get our current place sorted ready to physically move which again won’t be easy. Third I need to make a decision which will affect where I live for the next 25 years. At the moment I am torn between the size of property and its physical location, I need to balance both of these and meet the requirements from the rest of the family.

I do think this is an achievable goal so wish me luck.

Progress in my career

I need to be careful how I phrase this in case past, current or future employers are reading. I have been working hard at my current role for over a year and I am starting to wonder what is next for me. I am not entirely sure what form this is going to take yet, I have a few ideas on the go and hopefully I can reveal more soon.

Another one that is very achievable, as long as I have it in the back of my mind as the year goes on progress should be made.

See more of family

I recently missed a family celebration due to the expansion of my own family and it reminded me that there are aunts, uncles and cousins that I haven’t seen in years and it would be nice to reconnect. Also as my own boys get older and are learning to interact it would be great if they could get to know my own family better.

I don’t have a clear idea how to achieve this one as it takes two but I am going to make more of an effort.

Celebrate 5 years of marriage

Wow how has it been five years since we got married? At some point in 2018 I will take time out from the usual and spend it with the wife away from the boys. I won’t give away more details as I want that to be a surprise.

Celebrate the birth of my two sons

I don’t want to get my boys christened as I would like them to make up their own minds when they are old enough, however I do want to celebrate all that they are with a thanksgiving service. Not sure when we are going to sort this, however it is my hope that it will be soon.

Family holiday

Not much of a goal but in the summer I will take time out to spend a week away with the family. In previous years we have done scotland, wales and northumberland. This year I am thinking of somewhere not too far away as Edward is still small, but maybe near where I grew up.

Lightning Talk

A lightning talk is a very short talk or presentation given at a conference or user group. Friends of mine are having lots of success talking about what they know about and I think it might be time for me to have a go. I am not a natural public speaker so this is a step out of my comfort zone, however I am going to start small and see what happens. At the moment I don’t know what my subject or topic is going to be, my suspicion is that it will be devops related as that interest me more but watch this space.