Hello and welcome to the CogBlog. Here you'll find the (mostly) informative ramblings of the CogWorks staff on life, the Universe and Everything. Cogworks http://thecogworks.co.uk.aspx Fri, 10 May 2013 10:47:00 Fri, 10 May 2013 10:47:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2013/may/employee-no-1-is-leaving-the-building…/ When Dan, Tim and myself started The Cogworks back in January 2006, we were 3 blokes in a basement on Drury Lane, London. We managed pretty successfully for a year or so, but then as with all small companies the workload and number of clients put us in that most difficult of all places. Do we struggle on through or do we employ someone? To employ or not to employ...that is the question!?! Of course running your own business means a constant stream of difficult decisions, but taking on a new member of staff who is not part of the original startup group takes the business to a whole different level. Its not just about 'you' anymore. You have responsibilities, someone else's mortgage to think about, their kids, wife, life etc. This all needs to be taken into account before your own personal situation, so it is not a decision we took lightly (and still don't!) A lucky contact! Anyway, we decided to do a bit of digging around and tentatively asked about on Twitter and the Umbraco forums for anyone looking for an Umbraco developer role. We were contacted kind of out of the blue by a guy named Ismail Mayat…you may know him from such packages as uComponents, Examine Inspector for Umbraco, CogUmbracoExamineMediaIndexer, Live Logger for Umbraco and a mammoth 2369 posts and 3133 Karma points!! Ismail seemed like a good fit for what we needed to help us grow and get the projects out the door. He has a great knowledge of a variety of different systems and technologies, has a passion to constantly learn, he was already an Umbraco expert, already established in the community and most importantly an all round nice bloke! So We decided to take the plunge and asked him to join us on our merry Umbraco mission. ...and it was definitely a risk worth taking and has paid dividends a-plenty. Ismail jumped in at the deep-end, grabbing projects, suggesting ideas, mucking in where needed and generally doing everything needed to get the job done. And its been a great 3 and a half (ish) years. We've grown from 4 to 10 people and are growing steadily with every year that goes by with larger clients and more projects. We've always encouraged Ismail, as we do with all our guys, to be a part of the community and get involved, culminating in his nomination (again!) this year as an Umbraco MVP! And now, the end is here... But now the time has come to say adios to Ismail. He has decided to move on to pastures new, with new challenges and some different opportunities and we say good luck to him! We're sad to see him go, but also happy that he has helped us along our journey. Hopefully we've help him grow and improve his skills and provided a fun, if somewhat disorganised (at times) working environment. But we won't be saying goodbye…just au revoir…I know our paths will cross again in the future and this is why I love the Umbraco community so much. Its like one big family! You get to meet a whole heap of great, talented people who all have a common goal and are generally really nice guys… Good luck anyway Ismail... http://thecogworks.co.uk.aspx/blog/posts/2013/may/employee-no-1-is-leaving-the-building…/ http://thecogworks.co.uk.aspx/blog/posts/2013/may/employee-no-1-is-leaving-the-building…/ Fri, 10 May 2013 10:47:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2013/april/organising-an-umbraco-festival/ I've been asked a few times recently about my experiences of setting up and running the Umbraco UK Festival over the past few years, so I thought I'd share a few thoughts via the medium of expressive dance, but as you can't see me I guess writing it up would be more useful... First a little history... The idea to start an Umbraco UK festival came after myself and Warren Buckley organised the Umbraco 5th birthday celebrations in London back in 2009. This was to mark the 5th anniversary of the open sourcing of Umbraco by Chief Unicorn, Niels Hartvig.  The day was a great success and the UK event attracted over 110 people to a very rocky boat in central London. It was by far the highest attended event out of a few that where held at the same time around the world. After some great talks and lots of birthday cake, I thought it would be a great idea to run some kind of regular Umbraco specific developer conference right here in the UK.  At that time there was only really the one and only Codegarden and unless you have a healthy bank balance and 4 days holiday to use up or your boss is in a particularly good mood, that is a pretty big commitment to make for most companies. And so the Umbraco UK Festival was born... Picking the right venue This has proven to be a surprisingly tricky task. We've had 2 tracks on at the same time for the past 3 events, and it has worked pretty well, but the main problem is finding a venue that has 2 rooms of equal size. If you have 200 people attending with 2 tracks, you need to have at least 2 rooms that can hold a minimum of 100 people. This is not as easy as it sounds. In my research in London specifically I've seen some great venues with main rooms that can hold up to 250 people…even up to 1500! But it is the size of the 2nd room that is the stumbling block. We've tried 3 different venues so far, each with their own challenges. In 2010 the 2nd room was tiny, which wasn't an issue due to the number of attendees, however the main room was also small so it was a bit of a squeeze for the main keynotes. 2011 saw a decent space for the main room, but the 2nd room was on the small side, and there was nowhere for people to breakout if they needed to make a phone call. 2012 was the best yet. We had 3 rooms, the 3rd being a small breakout room which came in very handy particularly for Anthony Dang to get some beauty sleep! The search for 2013 continues... Audio/Visual equipment Decent lapel radio microphones are a must. If you can't get these, then make sure you have some decent microphone stands! Also sensible, usable areas for presenters to stand i.e. lectern, high level desk, decent internet connection, all working and tested A/V connections etc I learnt the hard way. Not having enough microphone stands and a decent desk at the first festival, left Morten Christensen trying to type and hold a microphone whilst crouching down over his laptop on a chair, which was no easy task!! - sorry Morten :) Obviously projectors are a must have for the audience to see what the presenter is talking about. They may have a PowerPoint or a code demo with small text. So the biggest, highest resolution screens you can get are essential. Also, if the room is a large one, then secondary screens closer to the audience are useful. And most importantly, make sure you have tested all the projectors, microphones, screens and any other gadgets you plan to use. Things will go wrong, its inevitable! So always best to be prepared. Also. make sure you have a couple of spare Apple MacBook adapters lying around. They always come in handy! Video the event Videoing the event is very important especially if you do have 2 tracks. This way at least everyone can see the talks they missed afterwards. However, make sure the videos are of a decent quality. Unfortunately, the videos we have taken in the past (last year included) have been pretty erm…crap! So apologies for that, I'll make sure this years event is much better! Coffee and lunch I learnt my lesson the hard way! This year to save a bit of cash because the venue charged a 'per serving' rate, I decided to scrap the late morning coffee serving...and boy did I hear about it! We had people running around, crying, climbing the walls, calling the emergency services, leaving the venue, we almost had a riot on our hands…all because of lack of coffee! Even our very own Anthony Dang was so weak from lack of caffeine he had to have a little siesta in the 3rd room to recover! Lunch is also very important. It doesn't seem to matter what special guests you have, what amazing announcements you plan to give, get the lunch wrong and you will know about it ;-) So make sure you get it right, and make sure there is enough to go round too! Communication Make sure there are printouts of the running order for the day and what rooms they are in. Also make sure you have a decent description of what the talks are about. And most importantly of all, make sure you stick them to the wall so everyone can see them!! Otherwise you end up with a lot of confusion and a lot of questions. Talk Subjects Make sure the talks are interesting and relevant, however it doesn't hurt to have some non-umbraco but related talks as well. At last years event we had RedGate presenting some related developer products, plus a talk on Agile/Scrum techniques which proved very popular. We found that open spaces were also useful and this is a lesson learnt from the 3rd day at Codegarden. People like to participate, collaborate and discuss subjects. After event social Interesting talks, guest speakers and flashy presentations aside, one of the great things about Umbraco is the fantastic community. The after event social is a great way of unwinding and catching up with people, making new friends and generally having a good time. The last 2 years, has seen the event grow into a 2 day affair, there has been a mini-social on the Thursday night and then a main event on the Friday.  Its a good idea to reserve a space anyway and make sure there is food available and if you're lucky you may see Pete Duncanson and Lee Kellher 'bustin some moves' on the dance floor...a site not to be missed!! Well thats just a little insight for now…I've probably missed loads of stuff out but its all in the planning, so if you are looking to organise an event, just go for it. Its hard work, but ultimately rewarding and well worth the effort :) http://thecogworks.co.uk.aspx/blog/posts/2013/april/organising-an-umbraco-festival/ http://thecogworks.co.uk.aspx/blog/posts/2013/april/organising-an-umbraco-festival/ Mon, 29 April 2013 13:10:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/april/examiness-hints-and-tips-from-the-trenches-part-10-document-writing-redux/ Just a quick post showing another use for the document writing event. When it comes to working with Umbraco Examine I take the lazy 'index all' approach when it comes to fields. So in my ExamineIndex.config I do not have IndexUserFields node therefore Examine will add all the fields.  However once or twice I have been caught out by this approach. Bascially after setting up the index there has been a requirement to sort by a non standard field e.g ArticleDate.  So in the past I have had to go back and add all the fields then enable sorting on that one field e.g <IndexUserFields> <add Name="firstName" /> <add Name="lastName" /> <add Name="roleName" /> <add Name="ArticleDate" EnableSorting="true"/> </IndexUserFields>  However there is a quick work around for this. You can inject in the sortable field using Document writing event: var indexer = (UmbracoContentIndexer)ExamineManager.Instance.IndexProviderCollection[MyIndex]; indexer.DocumentWriting += indexer_DocumentWriting; /// <summary> /// used to get low level lucene document access /// </summary> /// <param name="sender"></param> /// <param name="e"></param> private void indexer_DocumentWriting(object sender, Examine.LuceneEngine.DocumentWritingEventArgs e) { /* for article doc types we need to inject in sortable article date or use update date directly into lucene this saves us having to mess around with examineindex.config where you have to put in each field then on field u want to sort put in sortable attribute */ if (DocTypeHasArticleDateForSorting(e)) { DateTime articleDate; if (e.Fields.ContainsKey(DocumentType.Article.ArticleDate)) { articleDate = DateTime.Parse(e.Fields[DocumentType.Article.ArticleDate]); } else { articleDate = DateTime.Parse(e.Fields["updateDate"]); } // in __Sort_articleDate the __ implies is not analysed therefore can be used to sorting. sorted means its retrievable //see page 43 lucene in action 2nd edition for full explanation of options var field = new Field("__Sort_articleDate", articleDate.ToString("yyyyMMddHHmmss", CultureInfo.InvariantCulture), Field.Store.YES, Field.Index.NOT_ANALYZED); e.Document.Add(field); } } I can now continue to be lazy ;-} http://thecogworks.co.uk.aspx/blog/posts/2013/april/examiness-hints-and-tips-from-the-trenches-part-10-document-writing-redux/ http://thecogworks.co.uk.aspx/blog/posts/2013/april/examiness-hints-and-tips-from-the-trenches-part-10-document-writing-redux/ Thu, 11 April 2013 11:40:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/april/review-of-stacked-13/ Yesterday I attended my first Stacked event. This is the second such event in the Northwest the common theme for the day was Azure. I have to say I was very very impressed with the event. The venue John Moores university and the quality of the speakers and topics discussed was second to none.  Oh and the swag was awesome and I won a book Pro Nuget which was a bonus.   The sessions were recorded so hopefully at some point they will be publicly available. A quick overview of some of the things covered: Windows Azure mobile services - Mike Taulty Although currently in beta the Azure mobile services looks amazing. Its literally point click and within minutes you can have a webservice setup ready to be consumed by mobile devices.   You query the data using OData and there are plenty of different client libraries out there to help you do that.  Before data is sent back and before data is updated in a table you can pre process using node.js within the Azure backend. Its a bit like having triggers on database tables but much much more powerful. In fact you can use the node.js editor to perform all kinds of server side logic. If you have an Azure account you can turn on mobile services and have a play. Windows Azure website - What they don't teach you in school - Maarten Balliauw Maarten did a deep dive into the architecture, development flow and command line tools available for Azure websites. He also covered project Kudu (opensource engine behind git deployment in Azure website) and deployment using git.  You can view Maarten's slides here . Kung fu migration to Sql Azure - Scott Klein Scott discussed the numerous ways you can move an existing sql server database into Sql Azure and how to get round some of the common errors encountered namely an export package containing stuff not supported by Sql Azure. Building your own Azure management portal - Richard Conway Richard discussed the rich API available that will enable you to create your own Azure management portal. He also showed code samples e.g creating your own Azure website / mobile site via a windows console application.  I'm not sure how relevant this stuff will be for me personally, but for anyone looking to automate creation of assets in Azure this talk would be very useful. The last talk was on Embedded devices which I did not attend, but will be viewing the video when its up. Looking forward to next years Stacked many thanks to the speakers and sponsers for putting on a thoroughly enjoyable event. http://thecogworks.co.uk.aspx/blog/posts/2013/april/review-of-stacked-13/ http://thecogworks.co.uk.aspx/blog/posts/2013/april/review-of-stacked-13/ Wed, 10 April 2013 08:31:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/march/usignalrlog4net-is-dead-long-live-livelogger/ I blogged recently about an Umbraco package I created USignalRLog4net  this project has now been renamed to LiveLogger. It does almost exactly the same thing as USignalRLog4net but with some enhacements you get it from our.umbraco and you can view a quick video on YouTube . Enjoy! http://thecogworks.co.uk.aspx/blog/posts/2013/march/usignalrlog4net-is-dead-long-live-livelogger/ http://thecogworks.co.uk.aspx/blog/posts/2013/march/usignalrlog4net-is-dead-long-live-livelogger/ Mon, 18 March 2013 09:37:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/march/examiness-hints-and-tips-from-the-trenches-part-9-secure-searching/ In this post I am going to cover how to handle search results when you are working with a site that has a mix of secure and public content. By default out of the box Umbraco Examine will not index Umbraco protected content. However you may have a content rich site that needs to provide a fully functioning search both for public and extranet members. So there are a few things that need to be done to enable this. Firstly you need to update the ExamineIndex.config file so for the index that you are interested in, enable supportUnpublished. <add name="ExternalIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine" supportUnpublished="false" supportProtected="true" interval="10" analyzer="Lucene.Net.Analysis.WhitespaceAnalyzer, Lucene.Net"/> This will now make this index support protected content. The next step is to inject into the index groups that have access to the content for this we need to make use of our old friend GatheringNodeData event . using System; using System.Text; using System.Xml.Linq; using Examine; using Umbraco_Site_Extensions.Helpers; using umbraco.BusinessLogic; using UmbracoExamine; using umbraco.cms.businesslogic.web; namespace Umbraco_Site_Extensions.examineExtensions { public class ExamineEvents: ApplicationBase { public ExamineEvents() { ExamineManager.Instance.IndexProviderCollection[Constants.CogWorksIndexer].GatheringNodeData += ExamineEvents_GatheringNodeData; } void ExamineEvents_GatheringNodeData(object sender, IndexingNodeDataEventArgs e) { InjectGroups(e); } /// <summary> /// munge into one field /// </summary> /// <param name="e"></param> private void InjectGroups(IndexingNodeDataEventArgs e) { var node = new umbraco.NodeFactory.Node(e.NodeId); if(umbraco.library.IsProtected(e.NodeId,node.Path)) { var groups = Access.GetAccessingMembershipRoles(node.Id, node.Path); var groupsAccess =new StringBuilder(); //we now have list of group ids that access to this content foreach (var group in groups) { groupsAccess.Append(group); groupsAccess.Append(" "); } e.Fields.Add("GroupAccess",groupsAccess.ToString().Trim()); e.Fields.Add("IsPublic","false"); } else{ e.Fields.Add("GroupAccess","0"); e.Fields.Add("IsPublic","true"); } } } So now in the index we will have a new field called GroupAccess which should have values like '1045' and '1080' where the ids correspond to member groups set up in members section if the page is not protected it will just have value 0. Now that we have this data in the index we can use it as a filter in our queries. So as part of your search code you will firstly determine if a person executing the search is logged in or not.  If they are logged in then using the membership api you can get the ids of groups they belong to. Next as part of your query you could use a filter like:- query.And().Field("GroupAccess",memberGroupId.ToString(); query.Or().Field("IsPublic","true"); If they are not logged in then your query will look like query.And().Field("IsPublic","true"); Media protect If you happen to be using the excellent Media protect package by Richard Soeteman then you would in your GatheringNodeData event make use of the media protect api to get the groups that have access to the media item you are currently indexing, MediaProtect.Library.AllowedGroups(nodeId, path); The filtering is performed in the same way as content. http://thecogworks.co.uk.aspx/blog/posts/2013/march/examiness-hints-and-tips-from-the-trenches-part-9-secure-searching/ http://thecogworks.co.uk.aspx/blog/posts/2013/march/examiness-hints-and-tips-from-the-trenches-part-9-secure-searching/ Fri, 15 March 2013 14:52:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/march/examineinspector/ Its finally here! The moment you have all been waiting for...The Umbraco Examine Inspector! Its now up on our.umbraco.org  so get it while its hot. Its basically a cut down web based version of the Luke for Lucene tool. The package will install a dashboard control into your developer section.   It will then let you query your examine indexes in the umbraco backend - great for debugging some of those pesky examine queries! Enjoy!!!  http://thecogworks.co.uk.aspx/blog/posts/2013/march/examineinspector/ http://thecogworks.co.uk.aspx/blog/posts/2013/march/examineinspector/ Fri, 15 March 2013 12:20:00 Thom Sanders http://thecogworks.co.uk.aspx/blog/posts/2013/march/responsive-day-out/ I attended the Responsive Day Out in Brighton last week. This was a one-day conference which was packed with speakers but lacking in many of the 'extras' which people often think they need to include when putting on an event--no lanyards, badges, sponsor stalls, wifi, competitions, heating… As a result it was cheap, cheerful, and a very thought-provoking experience. There was just a single track and each speaker was allotted twenty minutes with some even sharing a slot. After each batch of three slots, there was a short Q&A session followed by good length breaks. Well done to Clearleft and others involved in running such a great event. Talks were from a variety of designers and developers, many of whom I follow but there were a few people I'd not heard of before. My big take-aways from the day were: Content is (still) the most important thing. A site without interesting, well-structured, and findable content is not going to do well. Progressive enhancement is the only sensible way in a world of thousands of device types. Make sure your content gets presented, then build from there where possible (and justifiable). We as an industry have dropped the ball regarding website performance since the olden days where it was often a major concern. We need to get back on top of this. Fluidity should be paired with resolution independence. We already have formats and tools for this, we need to make more use of them. Responsive Web Design is not just about mobile phones. Input devices are proliferating alongside device form factors. We cannot just cater for mouse, finger, or keyboard any more. Perhaps the biggest thing to come out of the day was the feeling that although the term RWD has been around for a couple of years, most of us involved in the industry are still trying to pin down exactly what it means and how it affects our thinking and processes. The phrase 'winging it' seemed to crop up an awful lot. We are still experimenting with stuff--even more so than is usual in the web design/development arena--but that obviously adds interest and excitement to the field of work and the fact that so many people turned up in Brighton to discuss the opportunities we have is the sign of a healthy industry I think. The following are some notes from the day. I haven't written up all of the talks but these were the ones of most interest to me and/or with specific points which I wanted to make a note of. Where possible I have linked to further info and/or slidedecks. I will update this post as I find links.   Cutting The Mustard - Tom Maslen Tom talked about how the BBC categorises browsers by capability and how each category is supported. The BBC is publicly funded and needs to provide services to as many people as possible. This means supporting older and lower-specced browsers and devices. They essentially split browsers into two camps with a test they call 'cutting the mustard'. Currently the decision is made according to something along the lines of: if (document.querySelector && window.localStorage && window.addEventListener) { alert('Mustard cut!'); } All browser requestes are initially served a basic version of the page. This should have a simple DOM, use exclusively CSS2, and be lightweight. Making sure the recipient gets the copy in a sensible layout is crucial. If it is decided that the browser 'cuts the mustard' then a a JS application is then loaded and this adds in further features, more advanced CSS, etc. Progressive enhancement as opposed to graceful degradation Older slidedeck on BBC and RWD Cutting the mustard   Responsive Web Fonts - Richard Rutter Richard is from Fontdeck and talked about methods for loading webfonts. There are currently three main options: Display text only once all of the webfonts have loaded. This is the most common behaviour these days Only send webfonts to 'larger screens'. Other devices are told to use system fonts Display text immediately using basic fonts. Then update to use the webfonts once they have been loaded 1 Is not really a option. Various factors can cause the text to not render for a long period of time, during which your visitor has no content to read. We can make use of media queries to call stylesheets containing webfont definitions for more capable devices (2). Eg. <link rel="stylesheet" href="/css/fancy-font-stuff.css" media="all and (min-width:569px)" /> The Google WebFont Loader can be used to monitor the status of each webfont your are requesting. This tool appends a class to the body tag of your page showing the overall status of the fonts as well as individual classes for each font as they load. Various javascript events are also raised during the loading. In your CSS you can then reference these classes to display webfonts as they become available. Notes, examples   Asset Fonts - Josh Emerson Josh discussed moving away from bitmap images where possible to improve 'resolution independence'. SVG is suitable in some circumstances but custom fonts are the ideal solution. Glyphs can be created using a tool such as Fontforge or we can make use of online services such as Icomoon and Symbolset . As well as being scalable, fonts can contain only the icons/glyphs we need and can be used simply in design and prototyping workflows. Josh put out a simple weather application which uses an asset font. It is on Github . Slidedeck   Design Systems - Laura Kalbag Laura spoke on having a system for your project. Ensuring that the visitor's experience is tailored but also has elements in common with the same content viewed in a different manner (eg. on a phone and on a desktop). Her system relies on setting typography and base units first and using these as the basis for everything else such as grids, image dimensions, etc. Make use of shapes to create themes-the examples showed circles. Rather than numerous mockups, use style guides, style libraries, and pattern libraries. Slidedeck   Playing With Game Console Browsers - Anna Debenham Anna has been testing browsers on games consoles, televisions, and other devices which are often overlooked. She pointed out that this is worthwhile for us as designers and developers as these devices are often used by the young and the poor and they also have interesting input devices and display characteristics (eg. dual screen in a single device or single screen on device, mirrored to TV). As we move from the traditional desktop dominance, we need to gracefully support 'alternative' devices. Due to slow release cycles, consoles often have poor support for the latest standards. The browsers often tend to be allocated very small amounts of memory as they are not considered a primary function of the device. Slidedeck Console browsers library   The Anatomy Of A Responsive Page Load - Andy Hume Andy works at the Guardian and they have a big interest in improving page load performance. His first point was that large wins can be made by simply ensuring: You use gzip compression You set far future cache-control where relevant They break the page load into three stages: Basic HTML and CSS Enhancements Leftovers (ads, analytics, etc) Scripts should be loaded asynchronously. Modern browsers support the async and defer attributes of the script tag. For older browsers the scripts can be called via JS. They are very specific about who is served with webfonts. They are not a core feature, they are an enhancement. Due to the vagaries of browser cache implementations they only serve fonts to localStorage as base64 encoded strings. They also only support WOFF. They are implementing enhancements as stand-alone components which can be re-used. Modularity is a key to perfomant pages.   What's Next In Standards - Bruce Lawson Bruce covered some of the upcoming standards including: CSS Device Adaptation It doesn't make sense for viewport information to be part of HTML markup. This is now moving to CSS. Currently supported in IE10, Opera. Safari support is arriving. You can specify max and min widths/heights, zoom, orientation, etc. Viewport information can be linked to media queries. W3C CSS Device Adaptation Flexbox The third version of Flexbox is now gaining support. Allows controlling spacing and alignment. Can be independent of source order. CSS Flexible Box Layout Module Pointer Events A new proposed spec from Microsoft which will replace the existing mouse events. Intended to support mouse, keyboard, touch, stylus, etc in a single API. W3C proposed spec Images Bruce also talked about proposals for a new image tag along the lines of the HTML5 video tag. This would enable multiple sources to be specified, fallbacks, etc.   The Edge Of The Web - Paul Robert Lloyd Paul talked about designing for simplicity and using the constraints of the device. He referenced TV title and ident artwork form the early sixties as an example of this. Average page sizes have been growing at a massive pace. This is mainly down to an increase in image sizes as higher resolution displays are being used. There has also been some laziness on our part as broadband connections became the norm and we started to optimise less as a result. We are not making enough use of text-whether that is copy or image formats such as SVG. For 'functional' or 'interface' images we should be using asset fonts, SVG, CSS. For content images we need to consider reducing quality. We are often sending down images which are too high quality for most users and display scenarios. We should think of some imagery as purely enhancements which can be loaded conditionally according to the browser capabilities and performance. Should we also consider a lo-fi aesthetic as the designers did fifty years ago moving from established print to the new, monochrome, low-resolution TV medium? Slidedeck and bits http://thecogworks.co.uk.aspx/blog/posts/2013/march/responsive-day-out/ http://thecogworks.co.uk.aspx/blog/posts/2013/march/responsive-day-out/ Mon, 04 March 2013 16:17:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/march/usignalrlog4net/ Umbraco V6, the latest Umbraco release now uses log4net for logging.  Its a fantastic bit of opensource software and I have used it previously on numerous Umbraco and non Umbraco projects. Out of the box you get a file appender configured and installed. This will create a text log file in your app_data directory. There are other appenders available .  The logging framework is extensible and you can create your own appenders.  SignalR is all the rage these days so as a side project, I created a package which makes use of a SignalR log4net appender, this will log in realtime to your browser. You can get the package on our.umbraco.org and you can get a quick overview by watching this video . Enjoy!!! http://thecogworks.co.uk.aspx/blog/posts/2013/march/usignalrlog4net/ http://thecogworks.co.uk.aspx/blog/posts/2013/march/usignalrlog4net/ Mon, 04 March 2013 08:10:00 Tom Smith http://thecogworks.co.uk.aspx/blog/posts/2013/february/a-quick-guide-to-rollup-style-reporting-with-google-analytics/ Google Analytics is an excellent tool for monitoring website usage and gaining insight into content performance. Arguably it's the industry standard when it comes to hosted analytics platforms - especially as it's free. It can however, become a little tricky to manage multiple web properties across different domains. We often get asked how web admins can get a view of how multiple properties are performing as a whole. To do that, some form of rollup reporting is needed. Here is a quick example to illustrate our standard solution; Dave Clientson has two websites; www.davescakes.co.uk www.davesbiscuits.co.uk As anyone who has ever been involved in the Jaffa Cake argument will tell you, cakes and biscuits are two very different things. Dave's Digital Marketing Manager, Donald recognises this, and has split the web presence into two separate sites accordingly - each site specifically tailored to meet the specialist needs of the target market. The current Google Analytics setup therefore looks like this: This setup allows Donald to supply in depth reports and analysis for each site at the weekly board meeting. Donald has impressed Dave over the last few months with increasingly detailed presentations, including a demonstration of new real time analytics features, and funnel visualisations of PPC campaign goals & conversions. In short, Donald's prospects for a well deserved pay increase are looking good. However, Dave suddenly wants to know how both sites are performing overall. He has got used to high quality reporting with advanced data visualisation, so Donald cannot simply export some pageview data to excel, add it together and bodge up a graph. Palming off the task to the unpaid intern isn't an option either (on account of Gerald being both Dave's nephew - and totally incompetent). To secure the pay rise, Donald either has to embrace sleepless nights and caffeine abuse, or master rollup reporting. The concept behind rollup reporting is simple - data from all websites, subdomains and associated properties is "rolled up" into a single account that we normally call the "Aggregator". The aggregator account is then used in exactly the same way as a normal analytics account - however, the resulting reports are created from data summed across all web properties. Step 1 - Create an "Aggregator" property Under the "Admin" pane for the desired account (as shown above), click the "New Property" button. Create an aggregator account as follows, use any of the site URLs in the URL field The properties list now looks like the following Step 2 - Add field for aggregator user agent id to umbraco If you are using a content management system, such as Umbraco, you most probably have the Google Analytics user agent id for the site stored somewhere in the CMS. It is also a good idea to also add a field for the aggregator so this can easily be updated in future if needs be. The User Agent ID's can be found on the property admin pane; e.g. Step 3 - Update tracking code on sites Now the google analytics tracking code needs to be modified to push interactions into both the original account and the aggregator account. If you are using Umbraco, the following code snippet should do it for you - just update the field names to the appropriate Umbraco property aliases. <!-- Google Analytics --> <script type="text/javascript"> var _gaq = _gaq || []; _gaq.push( ['_setAccount', '<umbraco:item Field="googleAnalyticsAccountNumber" runat="server" recursive="true"/>'], ['_trackPageview'], ['t2._setAccount', '<umbraco:item Field="googleAnalyticsAggregatorAccountNumber" runat="server" recursive="true" />'], ['t2._trackPageview'] ); (function () { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script> Alternatively, if you are not using Umbraco, replace the tags in the above code block with the necessary user agent ID's directly. Step 4. Update any event tracking or e-commerce tracking codes Any event tracking or e-commerce tracking code will also need to be updated to push to both accounts. For example, the following code snippet raises an event when someone clicks on a link to a PDF; jQuery('a[href$=".pdf"]').click(function () { _gaq.push(['_trackEvent', 'Download Literature', 'Click', this.title]); }); this would become jQuery('a[href$=".pdf"]').click(function () { _gaq.push(['_trackEvent', 'Download Literature', 'Click', this.title]); _gaq.push(['t2._trackEvent', 'Download Literature', 'Click', this.title]); }); You may want to adjust the second line to also push the site hostname if needed as this could make more sense in aggregate reports. Step 5. Set up domain filters on aggregator property This is a useful step if the source of the interaction needs to be explicitly clear in the aggregator account. For example, if nothing is done, Google Analytics will treat "/contact" on both sites as the same page and will only display one corresponding line in reports. This might be useful behaviour if the content of the sites is very similar. However, if the situation is similar to Dave's, then the content is very different across both sites, so it makes sense to know where the referrals are coming from. This can be achieved by applying a filter to the aggregator property's default profile which will append the hostname to the location of all interactions. "/contact" interactions therefore become "www.davescakes.co.uk/contact" and "www.davesbiscuits.co.uk/contact" respectively. Firstly, navigate to the "filters" pane of the default profile on the aggregator property Click the "new filter" button and apply as follows This basically modified the request URL field of every interaction that comes into Google Analytics by appending the hostname to the front. We will therefore be able to see immediately whether this was a cakes or biscuits interaction. Conclusion Rollup reporting with Google Analytics can give content owners quick and powerful insight into how web presences are performing as a whole. The technique is compatible with existing advanced implementations that involve event tracking or e-commerce. Luckily Donald was able to master rollup reporting by Tuesday morning, and therefore had 4 days worth of high quality data & visualisations to show Dave at the Friday board meeting. Dave was impressed, one thing lead to another and Donald's pay rise was secured. Further reading This blog post is excellent for a few more advanced techniques on sub domains and e-commerce tracking. It is quite an old post now so the examples reflect the old analytics UI, however, the principles are the same. http://thecogworks.co.uk.aspx/blog/posts/2013/february/a-quick-guide-to-rollup-style-reporting-with-google-analytics/ http://thecogworks.co.uk.aspx/blog/posts/2013/february/a-quick-guide-to-rollup-style-reporting-with-google-analytics/ Tue, 12 February 2013 12:13:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/february/examiness-hints-and-tips-from-the-trenches-part-8-custom-indexing/ As you may or may not be aware there were a number of updates made recently to the our.umbraco.org website ( full list of changes here ). One of the updates was searchable documentation.   The documenation section on the our.umbraco site lives in github and there is a scheduled job which pulls down a bunch of .md files from github and copies them to the documenation folder. In order to make the documenation section searchable I wrote a custom indexer to index the .md files I also updated the search configuration files to include the documenation index in any searches.   There are basically 2 ways of writing an indexer in Umbraco Examine, there is an easy and a hard way. Both have their advantages and disadvantages. Method Advantage Disadvantage Create SimpleDataIndexer Quick and easy Runs on a scheduled basis Create new indexer based on LuceneIndexer Full control over configuration and indexing More involved coding required   Given the fact that the documenation does not change very often I went down the first route, namely to create a new indexer based on SimpleDataIndexer.   The process that pulls down the documentation files has a webhook that is triggered after the files have been succesfully retrieved. I tap into this webhook and fire an event to indicate that all files are ready for indexing. The code for the indexer looks like:-   public class FileIndexDataService : ISimpleDataService { public IEnumerable<SimpleDataSet> GetAllData(string indexType) { var config = FileIndexerConfig.Settings; var fullPath = HttpContext.Current.Server.MapPath(config.DirectoryToIndex); var directory = new DirectoryInfo(fullPath); var files = config.Recursive ? directory.GetFiles(config.SupportedFileTypes, SearchOption.AllDirectories) : directory.GetFiles(config.SupportedFileTypes); var dataSets = new List<SimpleDataSet>(); var i = 1; //unique id for each doc foreach (var file in files) { try { var simpleDataSet = new SimpleDataSet { NodeDefinition = new IndexedNode(), RowData = new Dictionary<string, string>() }; simpleDataSet = ExamineHelper.MapFileToSimpleDataIndexItem(file, simpleDataSet, i, indexType); dataSets.Add(simpleDataSet); } catch (Exception ex) { Log.Add(LogTypes.Error, i, "error processing file " + file.FullName + " " + ex); } i++; } return dataSets; } } //method to create the dataset public static SimpleDataSet MapFileToSimpleDataIndexItem(FileInfo file, SimpleDataSet simpleDataSet, int index, string indexType) { var lines = new List<string>(); lines.AddRange(File.ReadAllLines(file.FullName)); var headLine = string.Empty; var body = string.Empty; if (lines.Count > 0) { headLine = RemoveSpecialCharacters(lines[0]); lines.RemoveAt(0); body = RemoveSpecialCharacters(string.Join("", lines)); } simpleDataSet.NodeDefinition.NodeId = index; simpleDataSet.NodeDefinition.Type = indexType; simpleDataSet.RowData.Add("Body", body); simpleDataSet.RowData.Add("Title", headLine); simpleDataSet.RowData.Add("dateCreated", file.CreationTime.ToString("yyyy-MM-dd-HH:mm:ss")); simpleDataSet.RowData.Add("dateCreatedSearchAble", file.CreationTime.SerializeForLucene()); simpleDataSet.RowData.Add("Path", file.FullName); simpleDataSet.RowData.Add("searchAblePath", file.FullName.Replace("\\", " ").Replace(":", "")); simpleDataSet.RowData.Add("nodeTypeAlias", "document"); simpleDataSet.RowData.Add("url", BuildUrl(file.FullName)); return simpleDataSet; } As you can see not much code is needed to write a custom indexer.  The disadvantage that you have when creating an indexer based on SimpleDataIndexer is that you only have a method to index everything in one go. You do not have a method to update an individual item e.g. if just one item is created or updated how do we update the index? One possible method would be to use FileSystemWatcher class on the documenation directory, then on change determine which file has changed and then look for that file in the index. If it does not exist add it to the index. If it does exist remove it then re-add it with the updated data. The single node indexing code would look something like: /// <summary> /// Removes an entry in the search index that is related to the post provided by the parameter. /// </summary> /// <param name="doc"></param> private void DeleteIndex(MdDocument doc) { ExamineManager.Instance.IndexProviderCollection["CustomIndexer"].DeleteFromIndex(MdDocument.Id.ToString(CultureInfo.InvariantCulture)); } /// <summary> /// Updates an entry in the search index that is related to the post provided by the parameter. /// </summary> /// <param name="doc"></param> private void UpdateIndex(MdDocument doc) { var examineNode = doc.ToSimpleDataSet().RowData.ToExamineXml(doc.Id, "CustomData"); ExamineManager.Instance.IndexProviderCollection["CustomIndexer"].ReIndexNode(examineNode, "CustomData"); }     If you were using SimpleDataIndexer to index database tables, then for per record updates you could use sql triggers and on trigger run update/delete/insert index updates using sql server triggers and calling .net assembly  that contains your Umbraco Examine insert record code (Note not tried this myself but this is how I would try to do it.) As part of the documenation search updates I previously posted using tuples to replace switch statements and I used this method in the search results control for our.umbraco. http://thecogworks.co.uk.aspx/blog/posts/2013/february/examiness-hints-and-tips-from-the-trenches-part-8-custom-indexing/ http://thecogworks.co.uk.aspx/blog/posts/2013/february/examiness-hints-and-tips-from-the-trenches-part-8-custom-indexing/ Mon, 11 February 2013 09:09:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/february/examine-inspector-screen-cast/ Quick video showing the examine inspector in action you can view it here http://thecogworks.co.uk.aspx/blog/posts/2013/february/examine-inspector-screen-cast/ http://thecogworks.co.uk.aspx/blog/posts/2013/february/examine-inspector-screen-cast/ Fri, 08 February 2013 14:31:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/february/the-examine-inspector/ How many times have you had to debug an Umbraco Examine index using Luke and how many times have you wished you could just view the index through a web browser instead of having to open a Java application on the server or having to download the index locally then opening it using Luke? Well...now you can! Enter the Cogworks Umbraco Examine Inspector. Its a little side project that I am working on, basically its a cut down web based version of Luke. It will eventually end up as a dashboard control so you can then access your indexes from the Umbraco backend.  Some screen shots below: Overview   Search Index files Backup One future potential use for this tool is the ability to search umbraco logs. As of Umbraco V6 all logging bar audit trail is handled by Log4net. It shoud be pretty straight forward to write a Lucene/Examine appender, which will give you an index of all log entries, then using the Cogworks Umbraco Examine Inspector you will be able to search log entries using rich lucene syntax . It's still a bit rough round the edges, needs a bit more testing and some styling. What do you think? PS: No idea on release date but hopefully soon. http://thecogworks.co.uk.aspx/blog/posts/2013/february/the-examine-inspector/ http://thecogworks.co.uk.aspx/blog/posts/2013/february/the-examine-inspector/ Fri, 01 February 2013 15:27:00 Anthony Dang http://thecogworks.co.uk.aspx/blog/posts/2013/january/automation-sparta-team-city/ In this post I will talk about automatic builds and deployment to specific environments using Team City. This ties in directly with my previous post:  Automation Sparta - Multiple Deployment Code Repositories with Kiln Introduction Let's face it. Everyone is using Team City. What we love about Team City is being able to hook up configurations to different repositories, and to trigger builds. This enables us to automatically deploy when we check in code from any machine.  Our Team City configurations contain Dev, Staging and Live builds (see image below).    Multiple Version Control Settings In the previous image we see build configurations for Dev, Staging, Live environments. Each of these points to a different repository (see previous post for why). It is super easy to set these up (see image below). Just point your VCS in each build configuration to a different repository.    Build Triggers This is such a simple concept. When Team City sees a change in a repository, it automatically builds (see image below). All you need to do is add a trigger for each build configuration. Our Team City configurations all have deployment as the last step. This deployment step can use anything. eg. FTPS, Rococopy, WebDeploy etc. There are plenty of tutorials on the web involving Team City and deployment, so I won't go into it here. Once you have deployment as a build step, Build Triggers will be a massive time saver. No more logging into Team City to trigger a build. Just check in your code and make a coffee. By the time you come back, your site is deployed to your desired environment.   Note: We prefer to exclude build triggering for our Live build configuration. The reason is that we prefer to have more human attention when it comes to deploying to a live deployment.    Conclusion In this post I presented a Team City setup which involved: Multiple Build Configurations, each pointing to different code repositories. I showed that by adding a deployment step to your configurations, and build triggers, it is possible simply check in your code and have it automatically built and deployed to a desired environment. Enjoy! http://thecogworks.co.uk.aspx/blog/posts/2013/january/automation-sparta-team-city/ http://thecogworks.co.uk.aspx/blog/posts/2013/january/automation-sparta-team-city/ Thu, 24 January 2013 11:38:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-7/ One of the really cool things about searching with Google is the Google advanced search form . This gives you quite a few options to perform really powerful, granular searches.  You can also make use of operators in the search boxes. It would be really nice to be able to do something similar in Lucene. Querying in a similar way is possible using Lucene query syntax  however its not very user friendly. Thankfully there is a Lucene query constructor written in JavaScript by Kelvin Tan.  Its quite old circa 2005 and no longer under active development however with a few modifications its very powerful and can give us advanced search capabilities. You can view a demo at the bottom of this post.  The source code can be downloaded here   The source contains all the JavaScript files and two html pages. The first test2.html is the original written by Kelvin which gives you quite a few options and also contains documentation on how to modify the JavaScript files to influcene the actual query to be generated. The other file advanced.html is our hacked version. The main thing to note is in the file luceneQueryConstructor.js we have a field 'luceneFieldAliasPrefix'. Currently this is set to 'bodyText', however you can set it to another field if you so wish.   I would recommend using the 'GatheringNodeData' event to create one big content field which has all the field content in it and then using that as the main search field. Once you have generated the query you can pass the query string to Umbraco Examine using the raw query API call.  BaseSearchProvider searcher = ExamineManager.Instance.SearchProviderCollection["myindex"]; var criteria = searcher.CreateSearchCriteria(IndexTypes.Content); var query = criteria.RawQuery(generatedQuery); var results = searcher.Search(query.Compile()); Using the demo You can test the lucene query constructor below .  In all of the fields you can make use of wildcards and fuzzy search syntax provided you only have one word - see wildcard search syntax  and fuzzy search syntax . In the phrase field if I have more than one word and you want to make use of proximity or boost then ensure that the phrase is in quotes e.g Phrase operators: Operator Phrase syntax Proximity jakarta apache "jakarta apache"~10 Boost jakarta apache "jakarta apache"^4   I have not fully tested all the possible queries that can be generated but I have tested quite a few and they work really well. NB: The demo does not perform any searches it just generates a Lucene query string. http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-7/ http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-7/ Wed, 23 January 2013 15:30:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-6/ In this episode of Umbraco Examine hints and tips from the trenches, I am going to cover working with facets using BoboBrowser. Faceting or faceted search, also called faceted navigation or faceted browsing, is a technique for accessing information organised according to a faceted classification system, allowing users to explore a collection of information by applying multiple filters. It is effectively a search within a set of search results. For example performing a search for 'Lord of the Rings' on Amazon will give you a list of results but on the left hand navigation menu it will also give you the facets. Clicking on these facets will filter your results. LinkedIn also makes use of facets and a significant percentage of LinkedIn is powered by Lucene. A custom faceting library called BoboBrowser had been built for this purpose. The library is opensource and has been ported to Microsoft.NET and can be found on codeplex I have created a wrapper class around bobo called FacetGenerator this makes use of the bobo api. As an end user you just need to instantiate the FacetGenerator class and pass a few things into the constructor:  string indexDirectory = @"E:\code\mysite\App_Data\ExamineIndexes\directory\Index"; //needs to be examine format query you can get from examine string query = "{ SearchIndexType: , LuceneQuery: +nodeTypeAlias:directoryitem -umbracoNaviHide:1 +contents:antique* }"; var fields = new List<string> { "directoryType", "countryName", "region" }; var facetGen = new FacetGenerator(atgDirectory, query, fields, 1, "contents", "directoryType"); var facets = facetGen.GenerateFacets();   This code returns a dictionary of type Dictionary<string,IEnumerable<BrowseFacet>> where the key is the facet. You need to pass into the constructor the Umbraco Examine index, the Umbraco examine generated query (this can be obtained from the Umbraco Examine search api e.g. var criteria = searcher.CreateSearchCriteria(IndexTypes.Content); IBooleanOperation query = criteria.NodeTypeAlias("NewsItem"); query = query.Not().Field("umbracoNaviHide", 1.ToString()); var results = searcher.Search(query.Compile()); criteria.ToString(); //gives you Umbraco Examine query string   Within the term generator class I convert the Umbraco Examine string to lucene query string. You also need to pass a list of fields that you want to generate facets for, the minimum facet hit value, the lucene index field your original query used and finally a field from the list of fields so that we can create an initial browser request to bobo browser. Before delving into the generator class that uses bobobrowser, some concepts to cover: BrowseSelection: A selection or filter to be applied, e.g. Color=Red FacetSpec: Specifies how facets are to be returned on the result object, e.g. Top 10 facets of car types ordered by count with a min count of 5 BrowseRequest: A set of BrowseSelections, a keyword text query, and a set of FacetSpecs. BrowseFacet: a facet, (a string value with a hit count) FacetCollection: A Collection object of BrowseFacets BrowseResult: Result of a browse operation. FacetHandler: A plugin into the browse engine to knows how to manipulate facet data. BoboIndexReader: A Lucene IndexReader containing a List of FacetHandlers.  The code for FacetGenerator is as follows: using System; using System.Collections.Generic; using System.Linq; using System.Text; using Lucene.Net.Analysis.Standard; using Lucene.Net.Store; using Lucene.Net.Index; using BoboBrowse.Api; using Lucene.Net.QueryParsers; using Lucene.Net.Search; using BoboBrowse.Facets.impl; using Lucene.Net.Analysis; using System.Diagnostics; using BoboBrowse.Facets; namespace Facets { public class FacetGenerator { string IndexPath; string LuceneQuery; IEnumerable<string> FieldsToFacetOn; int MinTermFrequency; string LuceneFieldQueried; string BrowseSelection; /// <summary> /// /// </summary> /// <param name="indexPath">full path to lucene index</param> /// <param name="luceneQuery">query that was initially run needs to be examine format</param> /// <param name="fieldsToFacectOn">list of fields we want to generate facets on</param> /// <param name="minTermFrequency">minimum frequency ie want for than 2 facets per hit</param> /// <param name="luceneFieldQueried">can be any field in index need it to build query object</param> /// <param name="browseSelection">one of the fields in initial list need it to actually do a browse</param> public FacetGenerator(string indexPath, string luceneQuery, IEnumerable<string> fieldsToFacectOn, int minTermFrequency, string luceneFieldQueried, string browseSelection) { IndexPath = indexPath; LuceneQuery = luceneQuery; FieldsToFacetOn = fieldsToFacectOn; MinTermFrequency = minTermFrequency; LuceneFieldQueried = luceneFieldQueried; BrowseSelection = browseSelection; } public Dictionary<string,IEnumerable<BrowseFacet>> GenerateFacets(){ var facets = new Dictionary<string,IEnumerable<BrowseFacet>>(); Directory idx=null; IndexReader reader = null; BoboIndexReader boboReader = null; try { idx = FSDirectory.Open(new System.IO.DirectoryInfo(IndexPath)); reader = IndexReader.Open(idx, true); var handlerList = GetFacetHandlers(); boboReader = BoboIndexReader.GetInstance(reader, handlerList); var browseRequest = new BrowseRequest { Count = 10, Offset = 0, FetchStoredFields = false }; // add a selection one of the fields in handler list var selection = new BrowseSelection(BrowseSelection); browseRequest.AddSelection(selection); browseRequest.Query = GetQuery(LuceneFieldQueried, GetLuceneQueryFromExamineQuery(LuceneQuery)); var facetSpec = new FacetSpec { OrderBy = FacetSpec.FacetSortSpec.OrderHitsDesc}; if (MinTermFrequency != 0) { facetSpec.MinHitCount = MinTermFrequency; } foreach (var facet in FieldsToFacetOn) { facets.Add(facet,GetHits(browseRequest,facetSpec,boboReader,facet)); } } catch (Exception ex) { throw; } finally { //cleanup if (idx != null) { idx.Close(); } if (reader != null) { reader.Close(); } if(boboReader!=null){ boboReader.Close(); } } return facets; } private IEnumerable<BrowseFacet> GetHits(BrowseRequest browseRequest, FacetSpec facetSpec, BoboIndexReader boboReader, string facet) { browseRequest.SetFacetSpec(facet, facetSpec); // perform browse IBrowsable browser = new BoboBrowser(boboReader); BrowseResult result = browser.Browse(browseRequest); Dictionary<String, IFacetAccessible> facetMap = result.GetFacetMap(); IFacetAccessible facets = facetMap[facet]; browser.Close(); return facets.GetFacets(); } #region private methods private ICollection<FacetHandler> GetFacetHandlers() { ICollection<FacetHandler> handlerList = new List<FacetHandler>(); foreach (var field in FieldsToFacetOn) { var handler = new SimpleFacetHandler(field); handlerList.Add(handler); } return handlerList; } private string GetLuceneQueryFromExamineQuery(string examineQuery) { int position = examineQuery.IndexOf("LuceneQuery: ") + 13; int noOfCharsToGet = examineQuery.Length - 2 - position; string luceneQuery = examineQuery.Substring(position, noOfCharsToGet); return luceneQuery; } private Query GetQuery(string fieldName, string queryString) { // get all directory items with name Gallery var parser = new QueryParser(Lucene.Net.Util.Version.LUCENE_29, fieldName, new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_29)); var q = parser.Parse(queryString); return q; } #endregion } } One thing to note the SimpleFacetHandler expects fields that are not tokenised and contain one value. So for example if you have a field in your index called directoryType that has value "Antique dealer" then by default Umbraco Examine will store it tokenised so you will end up with 2 facets "Antique" and "dealer", however you can update your Umbraco Examine config and add attribute EnableSorting=true and this will create a new field called '__directoryType' which will store an un-tokenised value which and you can pass this untokenised field into the term frequency generator. A Visual studio 2012 project zip file can be downloaded from here . You will need to update the index paths and field aliases to match your index.     http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-6/ http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-6/ Thu, 17 January 2013 11:05:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-5/ In this installment of the Umbraco Examine hints and tips series we are going to look at Umbraco Examine/Lucene index backups. When you want to create a backup of a Lucene index you cannot always just copy the index files. If an IndexWriter object is still open then the resulting backup copy can easily be corrupted as the index can change during the copy action.  One other point to note, if an IndexReader is active on the index while you are attempting to copy you will end up copying files that the reader is holding but are no longer current.  Finally the I/O load of the copy can slow down searching. Its not all bad news thankfully the Lucene api provides a mechanism to take hot backups so while the backup is taking place reading / writing can continue. Some code on how to take a hot backup:   const string indexDirectory = @"E:\someWebSite\App_Data\TEMP\ExamineIndexes\CustomIndexSet\Index"; SnapShot(); public void Snapshot() { StandardAnalyzer standard = new StandardAnalyzer(); var directory = FSDirectory.Open(new DirectoryInfo(indexDirectory)); SnapshotDeletionPolicy snapshotter = new SnapshotDeletionPolicy(new KeepOnlyLastCommitDeletionPolicy()); IndexWriter writer = new IndexWriter(directory, true, standard, snapshotter); IndexCommitPoint cp = snapshotter.Snapshot(); try { var files = cp.GetFileNames(); foreach (var file in files) { Console.WriteLine(file); } } catch (Exception ex) { } finally { snapshotter.Release(); directory.Close(); writer.Close(); } } The snapshot method makes use of the SnapshotDeletionPolicy object and then via the IndexCommitPoint object gets a list of files that need to be copied for a backup.  I am looping through those files and just writing out to the console, however you would need to copy those files to a folder or zip file.  This folder or zip file containing the index files will give you the backup.   How to restore index from backup The following steps will allow you to restore a backup: Close any existing readers / writers in a website. This is usually done by killing the app pool Remove all existing files from the index directory if you get access is denied then step 1 has not worked and you still have open reader / writer on your index Copy all files from your backup to the index directory. Thats it! More often than not when something has gone wrong with an index its usually just quicker to rebuild using something like Index Admin dashboard or Umbraco Examine dashboard . However when running either one of these tools they delete the existing index then rebuild. If they refuse to build because you have an error e.g GatheringNodeData event error and that is preventing rebuilding then you are stuck and this is where a backup would be very useful.  Another useful scenario for taking backups is taking a backup from a live site to debug a search issue which you cannot index on your machine becuase your index is out of date. http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-5/ http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-5/ Fri, 11 January 2013 09:13:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-4/ Continuing with my Examiness series today I am going to cover how to create a frequency term generator. Its not exactly Examine more Lucene but I think it ties in nicely with the Examine series.  The class will for a given index and specified fields in that index give you occurances of terms in the index. The question arises why would you want to know term frequencies?  Well a good example would be if you had a forum application you could use the frequency term generator to create a cloud tag showing 10 ten keywords. In fact we recently had a client who had a forum setup and they were displaying top 10 keywords on the home page thus directing users to current discussion themes, however the functionality had been built using node factory. Before delving into code we need to cover some deeper Lucene functionality namely Term Vectors.  There is an excellent chapter in Lucene in Action 2nd Edition on page 192 called "Leveraging Term Vectors" but in essence a term vector is a collection of term frequency pairs.  The Lucene api will allow you to retrieve a term vector for a given document and field in that document.  You can then add up the term occurance. The first step is to acquire a list of document ids ( NOTE: By document ids I mean lucene document ids not Umbraco ids. If you open a lucene index using Luke you will see the id field its like a database primary key assigned by Lucene when a document is added to the index).   You can get this list by executing a search using the Lucene api (could potentially use Examine but we need the document id which the ISearchResult object does not return although it may be in the fields collection of ISearchResult), some code: const string indexDirectory = @"E:\code\somesite\App_Data\TEMP\ExamineIndexes\CustomIndexSet\Index"; var directory = FSDirectory.Open(new DirectoryInfo(indexDirectory)); //get all document ids for given forum id var documents = GetDocIdsByForumId(6600, directory); private static List<int> GetDocIdsByForumId(int id,FSDirectory directory){ //create an index searcher that will perform the search Lucene.Net.Search.IndexSearcher searcher = new Lucene.Net.Search.IndexSearcher(directory); //build a query object Lucene.Net.Index.Term searchTerm = new Lucene.Net.Index.Term("forumid", id.ToString()); Lucene.Net.Search.Query query = new Lucene.Net.Search.TermQuery(searchTerm); //execute the query var topDocs = searcher.Search(query,searcher.MaxDoc()); _collector = new AllHitsCollector(topDocs.scoreDocs); var documents = new List<int>(); for (int i = 0; i < _collector.Count; i++) { documents.Add(_collector.GetDocId(i)); } searcher.Close(); return documents; } The AllHitsCollector class is taken from Examine code see here for code . You can read up on collectors in the Lucene in action book. Now that we have a list of document ids we can using a Lucene indexreader extract term vectors. So to create an index reader and get term vectors: var directory = FSDirectory.Open(new DirectoryInfo(indexDir)); reader = IndexReader.Open(directory); //loop through our doc ids foreach (var doc in sourceDocIds) { var termFreqVector = reader.GetTermFreqVector(doc, "body"); }    The term vector class has two methods that we are interested in: var terms = termFreqVector.GetTerms(); //actual terms var termfreq = termFreqVector.GetTermFrequencies(); //no of times they occur The methods return 2 arrays both of same length. You can loop through both of these and build up list of terms and frequencies. You can download full working version of the term frequency generator class from here . Usage code: List<KeyValuePair<string, int>> topTerms=new List<KeyValuePair<string,int>>(); //stopwords = list<string> of stopwords i.e. words you want to ignore //documents = list<int> document ids of lucene documents you want to iterate over to get term frequencies //indexDirectory = full path to lucene index // fields = list<string> lucene fields that contain terms // MinTermLength = the minimum length of term that we are interested in using (TermFreqGenerator mapGen = new TermFreqGenerator(stopwords, documents, indexDirectory, fields, 3)) { topTerms = mapGen.GetTopTerms(10); } foreach (var item in topTerms) { Console.WriteLine(item.Key + " " + item.Value); }  There is also a second constructor where you can pass in a delegate so that you can do your own filtering e.g. using (TermFreqGenerator mapGen = new TermFreqGenerator(stopwords, documents, indexDirectory, fields, 3,CustomFilter)) { topTerms = mapGen.GetTopTerms(10); } public static bool CustomFilter(string term, string field, int frequency) { //Console.WriteLine(string.Format( "term :{0} for field {1} has frequency: {2}",term,field,frequency)); if (frequency > 1) { return true; } return false; }  In this example I am only interested in anything that has term frequency greater than 1. This runs fine on an index with over 1000 documents however I suspect with bigger indexes it will not be as performant.  If you do use this class make sure you cache your results and have some kind of mechanism in place to invalidate the cache when the index is updated. http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-4/ http://thecogworks.co.uk.aspx/blog/posts/2013/january/examiness-hints-and-tips-from-the-trenches-part-4/ Fri, 11 January 2013 09:13:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2013/january/refactor-if-else-to-dictionary/ I recently had to work on some existing search code. The code had 4 search options giving in total 12 combinations.  Each combination would apply a different set of filters to some search results.  The main filter method was a block of 16 if else statements. Not too long ago I read a nice article on how to refactor switch case to dictionary . This article assumes you only have one lookup value, however in my situation there are 4.  So a quick google gave me a nice stack overflow article  which then gave me the idea to refactor to a dictionary using a tuple.  So some code to help explain things: Inititally we had if (searchWhere.Contains("documentation") && !searchWhere.Contains("project") && !searchWhere.Contains("forum") && !searchWhere.Contains("wiki")) { //documenation only searchResults = from r in searchResults where r["__IndexType"] == "documentation" select r; } else if (searchWhere.Contains("wiki") && !searchWhere.Contains("project") && !searchWhere.Contains("forum") && !searchWhere.Contains("documentation")) { //only wiki searchResults = from r in searchResults where r["__IndexType"] == "content" && r["nodeTypeAlias"] == "WikiPage" select r; } For brevity I have not included all 16 if statements.  So to cut down the amount of messy code, make things a bit more readable and probably makes things more efficient as a dictionary lookup will be quicker than going through a bunch of if else statements, I created a tuple dictionary with a method call as the value. /// <summary> /// wiki,forum,project,documentation /// </summary> static readonly Dictionary<Tuple<bool, bool, bool, bool>, Func<IEnumerable<SearchResult>, IEnumerable<SearchResult>>> lookup = new Dictionary<Tuple<bool, bool, bool, bool>, Func<IEnumerable<SearchResult>, IEnumerable<SearchResult>>>();   I then created an initialisation method passing all possible boolean combinations and functions I want to call for that set of values: /// <summary> /// init lookup dicitonary with all checkbox combinations and map /// the options to a filter method replaces the need for big if else /// </summary> private void InitLookUpDictionary() { if (lookup.Keys.Count == 0) { /// making assumption for truth table as = wiki,forum,project,documentation lookup.Add(Tuple.Create(true, true, true, true), value => All(value)); lookup.Add(Tuple.Create(false, false, false, false), value => All(value)); //single selections lookup.Add(Tuple.Create(true, false, false, false), value => WikiOnly(value)); lookup.Add(Tuple.Create(false, true, false, false), value => ForumOnly(value)); lookup.Add(Tuple.Create(false, false, true, false), value => ProjectsOnly(value)); lookup.Add(Tuple.Create(false, false, false, true), value => DocumentationOnly(value)); //doubles lookup.Add(Tuple.Create(true, true, false, false), value => WikiAndForum(value)); lookup.Add(Tuple.Create(false, true, true, false), value => ProjectAndForum(value)); lookup.Add(Tuple.Create(false, false, true, true), value => ProjectsAndDocumentation(value)); lookup.Add(Tuple.Create(false, true, false, true), value => ForumAndDocumentation(value)); lookup.Add(Tuple.Create(true, false, false, true), value => WikiAndDocumentation(value)); lookup.Add(Tuple.Create(true, false, true, false), value => WikiAndProjects(value)); //triples lookup.Add(Tuple.Create(false, true, true, true), value => AllExceptWiki(value)); lookup.Add(Tuple.Create(true, false, true, true), value => AllExceptForum(value)); lookup.Add(Tuple.Create(true, true, false, true), value => AllExceptProjects(value)); lookup.Add(Tuple.Create(true, true, true, false), value => AllExceptDocumentation(value)); } }  To do the look up and run the related method I do  searchResults = lookup[Tuple.Create(searchWhere.Contains("wiki"), searchWhere.Contains("forum"), searchWhere.Contains("project"), searchWhere.Contains("documentation"))](searchResults);  In the initialise method I can clearly see which combinations map to which method thus making more readable code. http://thecogworks.co.uk.aspx/blog/posts/2013/january/refactor-if-else-to-dictionary/ http://thecogworks.co.uk.aspx/blog/posts/2013/january/refactor-if-else-to-dictionary/ Mon, 07 January 2013 13:48:00 Anthony Dang http://thecogworks.co.uk.aspx/blog/posts/2013/january/red-gate-deployment-manager-10123-review/ We're always trying to improve our processes at The Cogworks, and as part of that is trying out new technologies. Lately we've been trialling Red Gate Deployment Manager  (DM), so this article will be a quick review of what we thought of it. At the time of this article, DM's main rival in this space was  Octopus Deploy . Introduction   DM consists of a main program which sits on your deployment server, and agents which sit on the servers to be deployed to. It's purpose is to provide a web interface for deploying applications to different environments and servers. Like Octopus Deploy, it uses NuGet Packages to do the transfers. In our setup, we have Team City performing builds and NuGet packaging. Installation   It's as simple as click click click. There's not much more they can do to make it easier. The only time consuming task is installing the agents, then copying and pasting secure keys between agents and the manager, which is necessary but is less work than installing Web Deploy.  NuGet Since DM deals with NuGet packages it was necessary to get our Team City installation to create them. This was definitely the most time consuming part of getting DM up and running. Agencies tend to use things like xcopy, FTP, Web Deploy etc to deploy the output of their built solutions. So having to set up Team City to create NuGet packages was a bit time consuming since we had not used NuGet packaging before. But if you're familiar with creating NuGet packages you won't have a problem Interface First Impressions The interface is clean but lacks visual cues. My first impression was "so what do I do now?". If you're like me then you hate reading documentation. I'd rather have a wizard guide me through configuration. At the least, there definitely should be some visual cues which show you that certain things need configuring. The design is a bit too vertically spaced. I find myself having to scroll up and down a lot. And this is with only a single test project. We have 50+ clients. That means 50+ projects on the dashboard. Lots of scrolling! As a user, I want to be presented with the option to easily deploy a specific release. However I found it hard to find a deploy button. There were many places where I would have loved to have seen a deploy button.  Finally, I'm not sure what the "Tasks" page is for. It looks more like a Log section.  Documentation The documentation is clean and helpful, but there seems to be a bit of documentation about features that are not enabled in the UI. For example, DM deploys to a new folder with a version number. The documentation says that IIS will be automatically switched to the new folder. However, it did not, and I could not find any mention of it in any UI. It's early days so I imagine documentation will be reviewed regularly. Configuration   Configuration was for the most part quite simple and intuitive. Adding servers is simple. The UX of adding projects needs a little work, or at least more help text. Everything just works the first time which is actually amazing. But it would be good to have more visual guides (see Interface First Impressions) By default, the agent will deploy to: C:\ProgramData\Red Gate\DeploymentAgent\Applications\<MyEnvironment>\<VersionNumber> This is not configurable in the UI, and may be a deal breaker for some companies. I imagine this will be configurable in a future release. [UPDATE]: It turns out that this is actually configurable under a variables subsection of a project. However, as stated already, there is nothing that indicates to the user that it is possible, or even where the default location is. Who would find it useful?  Companies that do not already have a deployment process would benefit from it.  If you already use Web Deploy, FTP, or even xcopy, then you probably don't need it (yet). However, as your company grows, you will eventually need to keep track of what is deployed, and where. That would be its main benefit for you. I have been informed that the coming versions will have SQL deployment. This will be a major win for any company. Wish list   1. Deployment triggers. For example, automatically notice that a new package is available in the NuGet feed, create a release, and deploy it.  2. FTP. We have clients with their own servers, so installing a DM agent on their servers is not an option. Typically the only way we can deploy to those servers is via FTP, FTPS, SFTP (depending on the client). I would be great if DM supported basic transfers like this. We can of course run these as an MSBuild task in Team City, but that defeats the point of using DM. 3. Automatic NuGet packaging. It would be fantastic if DM watched a folder for changes, then created NuGet packages for you on the fly.  4. Granular configuration in the UI. I would love to have complete control over everything. Conclusions The software is still in its infancy, but it has some definite promise. There is some need for some TLC and general polishing. The installation process is painless. Some configuration options are currently not available in the UI (eg. deployment path). I spoke to Red Gate about everything I've mentioned in this article. They were very open to any suggestion that would help improve the product. So, I encourage you to at least try it out to see what the future has instore. And if you have feedback, you can let them know here . Bottom line : If you don't already have deployment in your processes, then you would benefit from it. If you do already have deployment in your processes then you probably don't need it. However, you should keep an eye out for future releases as they will include deployment of databases. This will be a major win for companies like ours.   http://thecogworks.co.uk.aspx/blog/posts/2013/january/red-gate-deployment-manager-10123-review/ http://thecogworks.co.uk.aspx/blog/posts/2013/january/red-gate-deployment-manager-10123-review/ Tue, 08 January 2013 16:00:00 Anthony Dang http://thecogworks.co.uk.aspx/blog/posts/2012/december/automation-sparta-multiple-deployment-code-repositories-with-kiln/ In this post I will talk about Development repositories. Specifically I will talk about multiple deployment repositories and how we use  Kiln  to this end. Kiln Kiln  is a fantastic online mercurial code repository.  What we love about Kiln: Clean interface  Grouping of repositories Great online tools TortoiseHg just works FogBugz integration REST API Kiln's REST API means that we can programatically create code repositories for any project. What ever Code Repository system you choose, make sure it has some way of doing the same. Kiln allows us to create projects which have multiple repositories. We use these repositories to add granularity to our deployment. Specifically each repository holds a code base for a specific deployment environment (see below).  Multiple deployment repositories    Consider these user stories: "As a developer, I want to drop my new code in an environment specific deployment box, so that it is built and deployed to that specific environment". "As a developer, when I try to push my code to a deployment box, I want to know if it conflicts with another developer's changes. This will allow me to know that I should talk to the developer in question about whether I should merge his changes or not". "As a developer, I want to know what features have been pushed to our various environments, and if needed, roll back those deployments easily". Developers familiar with working with SVN will typically have a Trunk, working Branches, and Tags. The Trunk is typically representative of what is currently deployed on the Live (Production) environment. It is considered best practice to have working branches for new features. Many developers merge into the Trunk and then deploy from the Trunk. Lately there has been a move by some developers to using a Deployment branch. This way, the Trunk will only represent what is Live, and the deployment branch represents what is ready for deployment.   The problem with this approach is there is no way of knowing what features are on the Development or Staging environments (see the above user stories).  We take the deployment repository a few steps further with Kiln. The image below is a typical code repository setup which we use for all projects. As you can see, (See below for explanation).     In our setup, the 2 main repositories are a main working repository (which we call Build) and a Live repository (a Trunk equivalent). The Build repository is analogous to a feature branch/build branch. In fact, as usual there can be multiple branches for features as you would do normally. And you would only ever clone from the Live repository. To help in deployment to specific environments, we have Development (Dev) and Staging repositories. Think of Dev, Staging and Live repositories as boxes where code goes to get deployed. The fact that they are code repositories means that we have many more options, and much more power. Having seperate deployment repositories means that you can now automate deployment to your different environments. All you need to do is set up a build trigger on your Continuous Integration server (in our case Team City). For example, when a change in the Dev repository is detected, build and deploy. Automating your deployment in this way is extremely powerful. For example, when 2 developers try to push different features to the Dev server, the 2nd developer's push will fail. This is a signal to them that other code exists. They must communicate with the other developer or project manager about what takes precedence, or what should be merged. These multiple repositories may look strange at first, but if you think about it, we really only have a working repository, and a trunk (Live). The Dev and Staging repositories are just 2 more boxes which just help. Moreover, the extra boxes can be deleted.  Note : Regarding the third user story, it is possible to use other tools such as  Octopus Deploy  or  Redgate Deployment Manager  to track deployment versions. In fact, we are looking into integrating a deployment manager into our setup. Conclusion Multiple deployment repositories gives you granularity in your deployment. They allow you to automate deployment to you different environments via build triggering on a Continuous Integration server. Using a version control system with a REST API such as Kiln allows the automated creation of these repositories.  Related link: http://anthonydotnet.blogspot.co.uk/2012/10/this-is-automation-sparta-multiple-deployment-repositories.html http://thecogworks.co.uk.aspx/blog/posts/2012/december/automation-sparta-multiple-deployment-code-repositories-with-kiln/ http://thecogworks.co.uk.aspx/blog/posts/2012/december/automation-sparta-multiple-deployment-code-repositories-with-kiln/ Fri, 21 December 2012 16:02:00 Anthony Dang http://thecogworks.co.uk.aspx/blog/posts/2012/december/automation-sparta-wildcard-dns/ In this post I'll talk about the power of wildcard DNS and how you can use it to reduce configuration time in your web deployment. In web development there are typically at least 3 environments (servers) in your Project Universe. Dev, Staging (QA), Live (Production). To test a web application we obviously need a domain to hit for each environment. Some agencies use: something-dev.yourcompany.com something-staging.yourcompany.com something-live.yourcompany.com Some use: something.yourcompanydev.com something.yourcompanystaging.com something.yourcompanylive.com Others use: something.dev.yourcompany.com something.staging.yourcompany.com something.live.yourcompany.com Note: "something" is typically be a project code or project name. If you take a look at the 2nd and 3rd example, you can see that for each project (web application) the only part of the domain that will change will be the prefix. This is where wildcard DNS can be applied. All you need to do is register the following wildcards with your DNS provider: *.dev.yourcompany.com *.staging.yourcompany.com *.live.yourcompany.com Just make sure that each wildcard points to the respective server. Now any prefix can be used for each environment. For example. thisissomethingrandom.dev.yourcompany.com will automatically point to your development environment. The benefit of this is obvious. It removes the need for configuration of DNS for each project. You can just deploy your site to the respective server. All you need to do is create an IIS site. And this can be automated!  This also means that you can deploy to your Live server early in the development process.  As an added bonus, you'll also notice that many software licenses are valid for sub-domains. This means that you can use the same license file for each environment. Of course you may still need a new license for the actual live domain. But at least you can still test your site on the live server before the real domain is registered/available. Wildcard DNS is a massive time saver. It is just another DNS entry. It shouldn't cost you anything extra. We reap the benifits of it every day.   http://thecogworks.co.uk.aspx/blog/posts/2012/december/automation-sparta-wildcard-dns/ http://thecogworks.co.uk.aspx/blog/posts/2012/december/automation-sparta-wildcard-dns/ Fri, 14 December 2012 11:03:00 Tim Saunders http://thecogworks.co.uk.aspx/blog/posts/2012/november/organising-an-umbraco-hackday-the-cogworks-way/ A few people have asked me to blog about our experience organising an Umbraco hackday and the approach we took. The hackday before the Umbraco UK festival was very successful in terms of the total number of bugs fixed in a single day, however that wasn't achieved without a bit of planning and pre-work. So here's my guide to organising a successful hackday. Go Hackathon team Go!! You can 'Bash those Bugs!' Focus We were clear about what we wanted to achieve in the hackday and that meant deciding up front. Our hackday was to fix bugs, but yours could easily be to create a package or work on a new feature in the core. By being focused it means you can work out some of the not terribly exciting bits before hand (like a list of user stories or bugs to be fixed). This saves a whole heap of time and stops too much discussion about what to do before you get started. Make it achievable It's easy to get carried away when setting the focus and coming up with a massive new chunk of functionality that you'd like to work on. However you need to ask yourself, can this be done in a single hackday? By done I mean can you develop it, test it, document it and release it! If you don't think that can be done in a single hackday then it's probably not a good candidate, because after the hack day the team will disperse and you're much less likely to get the time to finish it. Of course that doesn't mean you can't have big ambitions but maybe ensure you break the big task into smaller chunks that deliver some benefit, so if you only get 60% of them done there's still a sense of having delivered something of value at the end of the day. Get help from the Core We were helped greatly before we started by Sebastiaan Janssen who helped us get a prioritised list of small (achievable) bugs that needed fixing. By involving the core in the pre-planning you will reduce the risk of working on things that are already getting added/fixed or working on something that is unlikely to get accepted into the core. Make sure you have an expert or two Although hackdays are for everyone, experienced or not. It really helps if you have someone knowledgeable in the room. We were blessed with 3 core team members but 1 would easily be enough. They can answer questions point people in the right direction and sense check what's being done. They needn't be core devs, but someone who knows Umbraco intimately. Pico Scrum! We used a technique during the hackday that I've named "Pico Scrum". I based it on the principals of Scrum but in a highly compressed manner. The use of this technique helps keep everyone focused and gives regular points where if things are going wrong they can be raised and fixed. I'll blog a bit more about Pico Scrum another time, but here's an overview: Start with an ordered list (backlog) of bugs/tasks/user stories Break the room into teams of not more than 7 people (try and ensure one expert per team if possible!) Ask each table to pick the items off of the backlog they think they can achieve in 1hr Write the items on post its and stick them on the wall Set a timer for 1hr and tell everyone to start! During development any task should be reviewed/tested by another member of the team. Pair programming is easier, but a simple code review will do. When the hour is up, go round each table and ask: What's done? What are you stuck on (need help with)?, What do you want to drop (if it turned out to be too hard)? Take 10-15mins to chat about any problems, look through the backlog generally have a break. Then go back to step 3 and repeat! Before you start you'll need to decide on a definition of done. This is an agreement on what constitutes that the task is actually complete. For our hack day it was: Bug fixed Tested/reviewed by another dev Patch created and attached to the case on umbraco issue tracker. Tim 'Pico Scrum' Saunders looking jolly pleased with himself Patch not Pull One thing bearing in mind is that Sebastiaan recommended that for loads of small bug fixes it was easier to use patches rather than pull requests, as these were easier for the core to review and accept. Obviously for larger chunks a pull request is more appropriate. People are giving up time to do this One key thing to remember is people are giving up family, work or spare time to come to your hackday, so it's important that you use that time as productively as possible as no one wants to have a wasted afternoon. That's why we wanted to have a clear focus and rules to what we did so there was a clear benefit at the end of the day. Have fun Don't forget in the heat of trying to get stuff done that it's meant to be fun, so it doesn't really matter if people can't get a task complete or want to work in their own way. So above all be flexible and respond to the group as the day progresses. If you do manage to organise a hackday and use this approach, I'd be really keen to hear how it went and any tweaks you may have made to the process to make it better. Happy Hacking!   http://thecogworks.co.uk.aspx/blog/posts/2012/november/organising-an-umbraco-hackday-the-cogworks-way/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/organising-an-umbraco-hackday-the-cogworks-way/ Wed, 21 November 2012 17:54:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-3/ In the previous post we covered the extremely useful GatheringNodeData event. I would go as far as to say if you are using Umbraco Examine and not using GatheringNodeData event you are doing it wrong! Anyhow, in this post we are going to cover the less frequently used but also very useful DocumentWriting event.  This event provides lower level Lucene access and a typical scenario where you would want this access is to boost a document or boost a field.   Before discussing boosting we need to understand how Lucene ranks a set of search results.  When you perform a search each document is scored and then the result set is ordered by score. The scoring is calculated using the formula below,   score(q,d)   =   coord(q,d)  ·  queryNorm(q)  ·  ∑ (tf(t in d)  ·  idf(t) 2  ·  t.getBoost() ·  norm(t,d)) t in q   Lucene Practical Scoring Function A full explanation of the scoring algorithm can be found on the apache site  however the part of the algorithm on the far right namely norm(t,d)  encapsulates boosting factors. Altering the boost will alter the score and therefore ranking. What is boosting? Not all documents are equal. In some cases you may want to artificially give higher ranking to certain documents. By default all Lucene documents have a boost of 1.0 by changing a documents boost factor you can instruct Lucene to consider it more or less important with respect to other documents in the index when computing relevance. Some example scenarios that can be achieved using boosting: Person doc type. If they have an important job title and they need to appear at top of person search list e.g Senior partner in law firm must always appear first. Boost documents by age. Penalize older documents - useful for news and business documents. Boost based on unique views - you would need to know the time slots to base the ranking on e.g last month, last week Documents with more likes - custom like functionality where likes stored in database table. More likes the higher the boost Weighted tags - Documents tagged with tags with a higher weight ( see xfs - term selector for Umbraco ) To acquire low level Lucene access in order to boost, you need to wire up the DocumentWriting event for a given indexer. The code is very simliar to implementing GatheringNodeData however there is one slight change /// <summary> /// </summary> public class ExamineEvents:ApplicationBase { public ExamineEvents() { var indexer = (UmbracoContentIndexer)ExamineManager.Instance.IndexProviderCollection[Constants.ATGMDirectoryIndexerName]; indexer.DocumentWriting += indexer_DocumentWriting; } void indexer_DocumentWriting(object sender, Examine.LuceneEngine.DocumentWritingEventArgs e) { var field= e.Document.GetField("title"); if(title=="Partner"){ e.Document.SetBoost(1.5f); } } } After we obtain the indexer we cast it to UmbracoContentIndexer, this then gives us access to the DocumentWriting event and that takes a DocumentWritingEventArgs parameter which exposes the actual Lucene document. When boosting say a news article that has a date field called publicationDate you could try the following: private static float BOOST_TODAY = 1.5f; private static float BOOST_YESTERDAY = 1.3f; private static float BOOST_WEEK_AGO = 1.25f; private static float BOOST_MONTH_AGO = 1.2f; private static float BOOST_THREE_MONTH_AGO = 1.15f; private static float BOOST_SIX_MONTH_AGO = 1.10f; private static float BOOST_ONE_YEAR_AGO = 1.05f; void indexer_DocumentWriting(object sender, Examine.LuceneEngine.DocumentWritingEventArgs e) { var field= e.Document.GetField("publishedDate"); var docDate = DateTime.Parse(field.StringValue()); float ageBoost = GetAgeBoostFactor(docDate); e.Document.SetBoost(ageBoost); } private float GetAgeBoostFactor(DateTime date) { DateTime today = DateTime.Now; float boostFactor; if (date.CompareTo(today)>0) boostFactor = BOOST_TODAY; else if (date.CompareTo(today.AddDays(-1)) > 0) boostFactor = BOOST_YESTERDAY; else if (date.CompareTo(today.AddDays(-7)) > 0) boostFactor = BOOST_WEEK_AGO; else if (date.CompareTo(today.AddMonths(-1)) > 0) boostFactor = BOOST_MONTH_AGO; else if (date.CompareTo(today.AddMonths(-3)) > 0) boostFactor = BOOST_THREE_MONTH_AGO; else if (date.CompareTo(today.AddMonths(-6)) > 0) boostFactor = BOOST_SIX_MONTH_AGO; else if (date.CompareTo(today.AddYears(-1)) > 0) boostFactor = BOOST_ONE_YEAR_AGO; else boostFactor = 1; return boostFactor; } Sometimes boosting can cause unexpected results hence in some cases a bit of trial and error is in order!   http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-3/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-3/ Mon, 19 November 2012 13:45:00 Anthony Dang http://thecogworks.co.uk.aspx/blog/posts/2012/november/automation-sparta-standard-naming-across-the-project-universe/ In this post I'm going to talk about naming conventions and how we effectivley use them at The Cogworks in our "no configuration setup". Introduction Naming should be structured, semantic, and consistent across environments and projects. Naming convensions are very important (not just in web development). They allow us to see patterns and repetition in our work. Where there are patterns and repetition, it is easier to see what can and should be automated. Naming in your Project Universe consists of: Databases IIS website names and IIS application pools IIS website folder names Continuous integration configuration names Code repository. In our no configuration setup, naming is so consistent that a person should be able to deduce a development url, repository link, database name, visual studio solution file name etc. based on the project code or project name. What ever naming convention you choose, you should be consistent. And the best way to be consistent is to automate. Following are the conventions that we use for all projects. And all of this is automated! Project codes We use project code prefixes in all databases/folders/IIS/environments etc. They consist of human readable characters, and project id's from our project management system (PMS). For example, if a client company is called Microsoft and the project id (from the PMS) is 123, then the project code may be MICRO123. This example uses a 5 letter human readable prefix. You may also want to pad the begining of the id with 0's for consistency. Note that these project codes may not be applicable to your company. What ever you use, just be consistent. Database Naming Database names are in the format: <ProjectCode>_<DescriptiveWord>_<Environment> "DescriptiveWord" is a word that allows a developer to know immediately what the database is for. For example, Umbraco would be used for an Umbraco database. Environment is either Dev, Staging, Live etc. Examples: Database with Umbraco on Dev: MICRO123_Umbraco_Dev Database for a HR system on Staging: MICRO123_HR_Staging The environment suffix is very helpful when dealing with the databases directly. It helps to avoid confusion when dealing with databases from different environments. For example, at some point you may have 3 Sql Management Studio windows open to different servers. This environment suffix helps to know which database belongs where. You don't want to truncate a table in the wrong environment! Database Logins Logins are in the form <ProjectCode>_<Environment> Examples: MICRO123_Dev MICRO123_Staging IIS Website Names, bindings, application pools, and folder names IIS site names are in the form: <ProjectCode>.<Environment>.yourcomany.com Examples: Live MICRO123.live.yourcompany.com Staging MICRO123.staging.yourcompany.com Dev MICRO123.dev.yourcompany.com Continuous Integration Configuration Continuous integration configuration titles are in the form: <ProjectCode> - Project Title. The reason we add the project title is to make it human readable. Example: MICRO123 - The Project Title Code Repository Code repository names are in the form: <ProjectCode> - The Project Title. For example: MICRO123 - The Project Title We use Kiln as our respository. With consistent naming we can also form consistent predictable repository urls. Examples: /Code/MICRO123-Microsoft/Working/Build /Code/MICRO123-Microsoft/Deployment/Dev /Code/MICRO123-Microsoft/Deployment/Staging /Code/MICRO123-Microsoft/Deployment/Live Note: We will discuss this repository setup in a later post. The Project Universe Putting it all together we can now see that a Project Universe (Dev, Staging, and Live) will have consistent naming throughout: Database MICRO123_Umbraco_Dev MICRO123_Umbraco_Staging MICRO123_Umbraco_Live Database login MICRO123_Dev MICRO123_Staging MICRO123_Live IIS Website Name, binding, application pool, and folder name MICRO123.dev.yourcompany.com MICRO123.staging.yourcompany.com MICRO123.live.yourcompany.com Continuous Integration Configuration MICRO123 - The project title Code repository MICRO123 - The project title Code respository Urls /Code/MICRO123-Microsoft/Working/Build /Code/MICRO123-Microsoft/Deployment/Dev /Code/MICRO123-Microsoft/Deployment/Staging /Code/MICRO123-Microsoft/Deployment/Live Project Management System The project title Summary Consistent standardised naming makes the Project Universe easier to manage. It adds predictability and reduces thinking. As you can see from the example above, the consistent naming lends itself to automation. We only need to type a project code once. Moreover, because the naming is so consistent, a human can deduce any url, code repository location, database name, and anything else we predefine in any project. Related external post: http://anthonydotnet.blogspot.co.uk/2012/10/this-is-automation-sparta-umbraco-uk.html http://thecogworks.co.uk.aspx/blog/posts/2012/november/automation-sparta-standard-naming-across-the-project-universe/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/automation-sparta-standard-naming-across-the-project-universe/ Wed, 14 November 2012 11:21:00 Anthony Dang http://thecogworks.co.uk.aspx/blog/posts/2012/november/automation-sparta-a-formal-ish-definition/ This is the first in a series of posts outlining Automation Sparta - a no configuration setup for web development and deployment. In this post I will outline the concepts and technologies used to create our no configuration setup.  Naming Naming should be structured, semantic, and consistent across environments and projects. Naming should be so consistent that person should be able to deduce a development url, repository link, database name, visual studio solution file name etc. based on the project code or project name. Knowledge about environments Knowledge about environments should be complete. This opens up the possibility of automating the insertion of configuration about environments. As a minimum, knowledge of the following is required: Code repository location Continuous integration location Server IP's SqlServer path Website root path SMTP servers And all the credentials to access the above Credentials should be predefined Credentials should be predefined for (at least) databases at the start of the project. This opens up the possibility of inserting these credentials and other configuration (automatically) into:  Web.config  Transform/deployment scripts Database interaction scripts Automatic deployment to a specified environment Automatic deployment to a specified environment should occur when code is dropped into a specific repository for deployment A continuous integration server can be set up to trigger builds when changes in a repository are detected.  Deployment to the Development environment can be achieved by checking code into a Deployment repository specific to the Development environment. Likewise,  repositories exist for Staging, Testing and Live environments etc. Visual Studio solution templates  Visual Studio solution templates should be automatically transformed for each project. This allows the developer to just start coding. Programatically preparing a Visual Studio solution from a template requires the renaming of the following: Folder names File names Namespaces Project names Project references Database interaction Database interaction should never require SQL Management Studio. Scripts should be created for creating databases, and deployment of schema and data DNS DNS configuration should never be required. Once naming is standardised across project, DNS configuration can be automated away (see below).   What should be created/configured automatically What is most important is that credentials are defined without the need for manual configuration. In our setup, we automatically define database credentials at the start of the project. This allows the automatic insertion of these credentials into the web.config and transform scripts etc. The following is a list of components/services in the Project Universe where automatic configuration/creation is essential: Code repository Database & credentials (on all environments) IIS (on all environments) Continuous Integration configuration (for all environments) - With deployment on code check-in Transform scripts -  For example, connection strings and related transforms DNS Note: You may choose to do actual database and IIS creation at deployment time. Technologies We acheive full automation of our setup using the following technologies Kiln - Mercurial Tortoise HG Team City MS Build Redgate SQL Compare & SQL Data Compare Wildcard DNS In the following posts I will discuss how we use these technologies to acheive the above automation. I will also give code samples where appropriate.    http://thecogworks.co.uk.aspx/blog/posts/2012/november/automation-sparta-a-formal-ish-definition/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/automation-sparta-a-formal-ish-definition/ Fri, 09 November 2012 21:50:00 Tim Saunders http://thecogworks.co.uk.aspx/blog/posts/2012/november/from-0-to-umbraco-using-azure-websites/ Azure has had a major facelift in recent months and is looking more and more compelling by the day. One of the new features is Azure Websites. These are currently in preview but basically you can spin up a new hosted website in a matter of minutes. Not only that you can even select Umbraco as a the type of website you want to create! Anyway let's take a look at how to do it. First up head on over to  http://www.windowsazure.com and sign up for a free trial. Once you've done that the first step is to enable the websites feature. It's currently in preview so you have to turn it on. Click on: Account and the click on: preview features. Scroll down and click the try it now button next to web sites! It'll whirr away for a bit and email you when the feature is enabled. When it's finished click on the manage link (or go here:  https://manage.windowsazure.com ). You should now see the websites option in the left menu. Click on that and choose create a website. Select from gallery (this is where you get to choose an Umbraco site!), scroll down and choose umbraco Click the next button. You now need to generate a url and configure a few Azure options, such as Region and DB password. The SQL password needs to be complex. More than 8 characters, wither a mixture of letters numbers casing and special characters i.e. ThisIsAP@ssword123 Click next again and you'll need to tell it to create a new SQL Database Server. Pick a login name and another complex password. Azure will now create your site! This will take a few minutes. When it's finished you can click on the URL and.... You'll see the standard Umbrco install screen! Go through the steps as normal (the DB settings are all pre-configured!) Select your starter kit and Boom! You've gone from 0-Umbraco in minutes! There's a ton more you can do with Azure sites, such as git deploy and scaling but that's for another blog post! I'll definately be using it when I need to spin up demo's or to try out Umbraco packages etc. It looks like a real time saver. It may even be the future of small scale Umbraco hosting. http://thecogworks.co.uk.aspx/blog/posts/2012/november/from-0-to-umbraco-using-azure-websites/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/from-0-to-umbraco-using-azure-websites/ Wed, 07 November 2012 18:04:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-2/ In part 1 of my Examiness series we covered useful tools, in part 2 I am going talk in depth about the GatheringNodeData event. Umbraco Examine has a rich event system and one very useful event to tap into is the GatheringNodeData event.  This event exposes a dictionary of fields that are going to be added to the index, you can also add your own fields to the dictionary or update / remove existing fields. This event is very useful when you need to massage the data before Umbraco Examine 'inserts' into the Lucene index. Some typical usages of GatheringNodeData to massage data: Merge into one contents field Create searchable path Perform 'select *' type queries Merge into one contents field The node data fields that you want to add to an Umbraco Examine index are controlled by the IndexUserFields element in the ExamineIndex.config. If this element is left empty then all fields in a node are inserted into the index.  In code to search on fields you would typically have a query like:- var query = searchCriteria.Field("nodeName","hello").Or().Field("metaTitle","hello").Field("metaDescription","hello").Compile(); This query can become quite lengthy if you have quite a few fields, additionally if you add a new field to a document type and you update the ExamineIndex.config to include that field you will need to update the code to also search on that field. However if you merge all the fields into one new contents field then the search query looks like:- var query = searchCriteria.Field("contents","hello") This is much shorter than the original query, also adding new fields is not an issue as the new field will be added to the contents field during indexing thus making it future proof. Create searchable path If you want to create search driven components like news listers then you can create Umbraco Examine queries to list all child documents of a particular parent, e.g. all news articles in the news section.  This would be done by searching for all documents that contain the parent news section document id in the path.   The path is stored in the index, however its stored as a csv list e.g:-   1,1056,1078 In order to execute a search to find all news items with parent 1056 in the path, we would need to massage the path data into a format that is searchable. So using the GatheringNodeData event we can replace the commas in the path with a space and inject into new field called searchablePath. We can then perform a search in this field and we will get our end result. Perform select * type queries When working with a database if you want to get all records in a table you can execute a query like 'select * from myTable'.  Lucene does not have a 'select *' equivalent, you must have a field and a value for that field to query on. One work around to enable 'select *' like functionality is to use the GatheringNodeData event to inject a common field into all records e.g create field called allRecords with value 1. You can then execute the following query:- var queryForAllDocuments = searchCriteria.Field("allRecords","1") This will return all records in the index. GatheringNodeData gotchas When working with this event ensure that you have plenty of try catch blocks with some logging so that any nodes that fail are recorded for future investigation. Without try catch blocks the whole rebuild index can sometimes fail, especially if you are doing anything with multi-node tree picker.  A common issue is when the node you are indexing has a multi-tree node picker and it is referencing a node that no longer exists. Some novel GatheringNodeData usages After my presentation at Umbraco UK festival  I was approached by Neil Tootell who showed me 2 really cool GatheringNodeData scenarios he recently implemented:- Using GatheringNodeData to get the HTML content of the current page and injecting the HTML into the index so that the HTML could be used for a preview overlay of the page in a search result set a bit like Google. Opening password protected PDF's by getting the password from the web.config to open the PDF How to implement GatheringNodeData To implement GatheringNodeData you need to create a class that inherits from ApplicationBase which can be found in the umbraco.BusinessLogic name space. You then get an instance of your index in the constructor of the class and wire up the event e.g using System; using System.Text; using System.Xml.Linq; using Examine; using Umbraco_Site_Extensions.Helpers; using umbraco.BusinessLogic; using UmbracoExamine; namespace Umbraco_Site_Extensions.examineExtensions { public class ExamineEvents: ApplicationBase { public ExamineEvents() { ExamineManager.Instance.IndexProviderCollection[Constants.CogWorksIndexer].GatheringNodeData += ExamineEvents_GatheringNodeData; } void ExamineEvents_GatheringNodeData(object sender, IndexingNodeDataEventArgs e) { AddToContentsField(e); } /// <summary> /// munge into one field /// </summary> /// <param name="e"></param> private void AddToContentsField(IndexingNodeDataEventArgs e) { var fields = e.Fields; var combinedFields = new StringBuilder(); foreach (var keyValuePair in fields) { combinedFields.AppendLine(keyValuePair.Value); } e.Fields.Add("contents", combinedFields.ToString()); } } }  In the next blog post we get deep and dirty with DocumentWriting event. http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-2/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-2/ Mon, 12 November 2012 10:49:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-1/ This is part 1 in my Umbraco Examine blog post series. Today I shall be covering useful tools.  Basically if you are doing anything with Umbraco Examine then you need the following: A copy of Lucene in action 2nd edition Luke for Lucene (java version)  ( .net version ) Lucene in action 2nd edition As you may or may not be aware Umbraco Examine is just an abstraction around Lucene. The Lucene in action book covers in detail the inner workings of Lucene e.g analysers, documents, fields. For detailed summary see Manning about page . The last few chapters cover detailed case studies with the LinkedIn case study being a real eye opener. In fact the LinkedIn case study pointed me to the Bobo browser faceting engine, I will cover Bobo in a future post. Luke for Lucene Luke for Lucene is the Lucene equivalent of SQL server enterprise manager. This tool is an absolute must when working with Lucene/Umbraco Examine. If you want to know what is in your index or test your queries then 'View the source Luke!'   An example on how to use Luke. Lets say you have the following code snippet that generates and runs an Umbraco Examine query. private BaseSearchProvider searcher = ExamineManager.Instance.SearchProviderCollection["MYIndex"]; protected void Page_Load(object sender, EventArgs e) { if (!IsPostBack) { ExecuteSearch(); } } private void ExecuteSearch() { var criteria = searcher.CreateSearchCriteria(IndexTypes.Content); IBooleanOperation query = criteria.NodeTypeAlias("NewsItem"); query = query.Not().Field("umbracoNaviHide", 1.ToString()); query = query.And().OrderByDescending("createDate"); var results = searcher.Search(query.Compile()); totalResults = results.Count(); SearchResultListing.DataSource = results.Skip(page * PageSize).Take(PageSize); SearchResultListing.DataBind(); //hidden field to store generated query generatedQuery.Value = criteria.ToString() + " count is " + totalResults; } This creates a query written to the hidden field that looks like  SearchIndexType: content, LuceneQuery: +(+__NodeTypeAlias:newsitem -umbracoNaviHide:1) +__IndexType:content You can cut and paste everything after the : into Luke and run that query to get some results.  Luke will also allow you to test your queries using different analysers. Here ends post no 1 part 2 covers every Examine users best friend GatheringNodeData event . http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-1/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/examiness-hints-and-tips-from-the-trenches-part-1/ Mon, 05 November 2012 07:35:00 Anthony Dang http://thecogworks.co.uk.aspx/blog/posts/2012/november/umbraco-uk-festival-automation-sparta/ Last week at the  Umbraco UK Festival  I presented a "no configuration setup" for web development and deployment. The technology has the ability to automate setup and creation of an entire Project Universe (Development, Staging, Live etc environments) in a single click. It also allows developers to download a completely configured Visual Studio solution, and automatically deploy to the various environments. The presentation was titled "This is Automation Sparta". In the demonstration of this setup, I created an entire Project Universe, installed Umbraco, and deployed to our Development and Staging servers, all in a matter of minutes. And all of this with no need for any manual configuration. Over the coming weeks I will be blogging about the concepts and technologies that allowed us (and will allow you) to create a "no configuration setup".      http://thecogworks.co.uk.aspx/blog/posts/2012/november/umbraco-uk-festival-automation-sparta/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/umbraco-uk-festival-automation-sparta/ Fri, 02 November 2012 11:40:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2012/november/umbraco-uk-festival-wrap-up/ Wow, well what a couple of days we've had! A big thank you to everyone who came along to the Umbraco Hackathon on the 25th and the main Umbraco UK Festival on the 26th October 2012. So nothing blew up, no-one got hurt and no animals were injured as far as we could tell, which makes the whole event a great success in our book! Although, admittedly there was a significant lack of coffee and a limited array of sandwiches on the day, but hey…we will learn by our mistakes, and using our own agile development processes, we will review, adapt and improve to make next years festival even better :) Hackathon Our first foray into our (trademarked) version of SCRUM, 'Pico Scrum' was a great success if we do say so ourselves. The initial aim was to fix 5 bugs, but in the first hour we had already fixed 6!! By the end of the day the total was 31 which was an immense effort by all involved. I'm please to say that all the fixes are already in the latest Umbraco release too! We had some great talks during the day. The main highlights were:- Project Belle The day started off with the unveiling of 'Belle', the codename for the new Umbraco UI, and as our friends from over the pond would say...'she sure is purdy!' Niels Hartvig and Per Ploug Hansen, went through a brief history of where the current UI came from and how it got to where it is now. Then he showed off some of the thoughts behind the redesign and the direction they are going. You can see the progress and get involved in the Umbraco UX Google Group . Contributing to the Core Newest member of the HQ and Umbraco veteran, Sebastiaan Janssen took us through the best ways of contributing to the Umbraco core either as bug fixes or new features. He's now going to be very busy keeping up with the flurry of patches and submissions he's likely to get!! In short, you can find all you need to know here -  http://our.umbraco.org/contribute Examiness (Hints and tips from the trenches) Lucene.NET and general search ninja Ismail Mayat, one of The Cogworks Senior Developers, held an in-depth talk on Umbraco Examine and things he can't do without. We will be posting many more posts about this, so keep your ears out for those. Automation Sparta Our very own Anthony Dang gave some great examples of how, at The Cogworks, we are starting to automate everything from the setup of IIS to build servers and deploying to live. Its automation, automation, automation here! The plan is to hopefully be so automated that we'll be able to work from home and run the company with a simple 'Next' > 'Next' > 'Finish' wizard process every morning! SCRUM at The Cogworks Oh that was me wasn't it! Well I waffled on for an hour or so taking people through how we have setup SCRUM and agile development practices at The Cogworks. I'll be writing about this too over the next few weeks so i'll let you know when they are ready. Relationships and communities (a designers perspective) Pete Hotchkin from Red Gate presented a look at relationships, both in open source and in development in general. He went through how some of the Red Gate developer tools such as performance profilers can actually help bridge that gap and get people working more harmoniously together. Other Highlights Tea Commerce - Anders introduced some new great features for Tea Commerce uCommerce - Soren talked though the new version of uCommmerce uSitebuilder - Pete Duncanson introduced how they use uSitebuilder for developing with Umbraco Load balancing - Tim Saunders went through the recommended setup scenarios and how other people may have done this as well uComponents - Lee Kelleher presented some hidden gems Community Presentations Due to a key speaker dropping out the day before, we had a slot to fill towards the end of the day. So I decided to open the floor up to the community. This proved to be a great opportunity for some exciting new developments to be presented. The first was Mentor Digital presenting their Web Blocks concept for editing in the back end. This looks very interesting especially with the new Umbraco UI on the horizon. More can be seen here:-  http://t.co/5YUHkX8G Niels Kühnel presented something he's been working on to allow huge amounts of data to be stored and indexed in Umbraco. I'm not sure exactly how it works, but it looked very exciting! Q&A The last session was a brief Q&A with Niels and Per which was a chance to ask them some questions about Umbraco, and also to delate matrimonial wishes…not sure who asked that, but that just shows how friendly the Umbraco community is :) What next?... We will be blogging in more depth about the day and some specific talks separately, so keep an eye out for those on twitter and Google+ over the coming weeks. You can see all the videos on the Umbraco UK Festival website . I am in the process of formatting them all so more will appear shortly. Apologies for the sporadic quality of them. We will make more professional ones next year. So all in all, the day was a huge success and everyone seemed to get something from it.  I hope to see some of you at Codegarden 2013 and it's (not so) little brother The Umbraco UK Festival 2013. See you next year!! http://thecogworks.co.uk.aspx/blog/posts/2012/november/umbraco-uk-festival-wrap-up/ http://thecogworks.co.uk.aspx/blog/posts/2012/november/umbraco-uk-festival-wrap-up/ Thu, 01 November 2012 08:37:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-examiness/ Well, another Umbraco UK festival over and done with and I can't wait for umbukfest13! It was great meeting up with so many old faces and plenty of new ones. For those of you who attended you will be aware that I presented a session on Umbraco Examine and how we use it at The Cogworks. There was some great feedback from festival attendees and I've received some additional awesome code examples from Neil Tootell (@toots) who is using the index for a really cool feature, but more about that in a future blog post. Anyhow I covered quite a few areas in my talk, so over the next few weeks I am going break it down and blog a little about selected topics.   Part 1 will cover some useful tools . http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-examiness/ http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-examiness/ Wed, 31 October 2012 13:28:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festivalthe-day-is-upon-us/ Well, the day is nearly up on us, and what a day we have planned. You can see the full line up of talks on the Umbraco UK Festival website . And don't forget we've also got some extra events planned around the main festival, so even if you can't make it to the festival itself, feel free to pop along to one of the following:- Pre-Festival Hackathon During the hack day we plan to work in small teams to achieve tangible results in a short space of time. The aim is productivity; we want to tackle any bugs and ideas that we think will take up to 1 hour to complete, then working in agile teams, tackle as many as we can! The achievements will be presented the next day at the Festival. Pre-Festival Meetup We've organised a pre-festival social on Thursday evening, so feel free to pop along if you're in or around Covent Garden. Post-Festival Social After the main day, we'll be winding down with a few bevoirs and some food in a local watering whole . So even if you can't make it to the main event, feel free to pop along and soak up the atmosphere.   http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festivalthe-day-is-upon-us/ http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festivalthe-day-is-upon-us/ Wed, 24 October 2012 13:48:00 Tim Saunders http://thecogworks.co.uk.aspx/blog/posts/2012/october/agile-development-the-importance-of-being-retrospective/ One of the first things we learnt when implementing agile is the power of the retrospective. It doesn't matter which form of Agile you are looking to implement, Scrum, Kanban, XP, DSDM the retrospective is possibly one of the most valuable tools in the box. If you don't know what a retrospective is, it's a look back at how things are going to find out what's working, what's holding you back and what you can do to improve. One of the key things to remember when implementing Agile is that it's not something with a fixed end point, only through constant learning and improvement will you attain an Agile state. (Reading that back it sounds rather like I've been sitting too long in a room listening to whale music, but you get the picture). This is why the retrospective is so valuable, as it acts to steer the course as you implement new ideas and techniques. Here's a simple template for how to hold a retrospective: Get the team in a room, free of distractions. Make sure everyone understands the Retrospective prime directive . This is an important step, particularly in organisations where there hasn't been a culture of trust or people may be nervous expressing themselves. Then ask the following three questions: What's going well, what are our strengths? What didn't go well, what are our weaknesses, what's holding us back? What can we do to improve? At the end of the meeting pick a few improvement areas to focus on before the next retrospective. Get them stuck in the wall somewhere, so everyone can see them. Then book a time for the next retrospective and thank everyone for their time and contribution! Challenges You try and make too many improvements at once It's easy to want to try and get all the improvements in place straight away. However this can lead to confusion and a lack of focus, so I strongly recommend just picking a few critical areas, or if you don't have much time just pick one! People are bored or disengaged This normally happens, if you have a retrospective a few times but nothing comes from it. The second most important factor of a retrospective is that it leads to action! If it doesn't people will see it as a waste of time and become disengaged. It seems to take a long time! Often the first few retrospectives can take a while, but they do get shorter I promise! If it's a real problem then introduce a time-box for the three questions, so you don't get bogged down on a single area. As always in agile some improvement is better than none. One final take-away point: Although this is how we conduct retrospectives, you may find that your organisation needs a different format, there are many ways to run a retrospective . As always inspect and adapt. So if you aren't doing a retrospective, then I strongly recommend you start, as before long continuous improvement will become second nature. http://thecogworks.co.uk.aspx/blog/posts/2012/october/agile-development-the-importance-of-being-retrospective/ http://thecogworks.co.uk.aspx/blog/posts/2012/october/agile-development-the-importance-of-being-retrospective/ Mon, 15 October 2012 15:52:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2012/october/dddnorth2-quick-review/ Saturday 13th October I attended the second DDDNorth.  It was held at Bradford University School of Management, the first one was held at Sunderland University.  Myself and three friends who attend my local thai boxing gym and who also happen to be IT geeks travelled across from Preston. A few people had recommened the DDD events in the past but I had never mangaged to get round to it. This year I did and I have to say I was very impressed.  The quality of the talks was of a very high standard I learned quite a lot.  I attended the following sessions: Lions and tigers and hackers! Oh my! Phil Winstanley.  This session was a real eye opener into the world of hacking.  The main take away point was to read  The Security Development Lifecycle  and incorparate into your development processes. There is also the Microsoft website regarding security development lifecycle see here Async C# 5.0 Liam Westley - This session was mostly based on the Microsft white paper on async in C# 5 . Defensive Programming 101 Nial Merrigan - This covered top 10 security issues that we have all been guilty of at some point.  The demo of how to download a web.config literally made my jaw drop.  In summary if you have web page that streams a file e.g pdf files and you have url like mystreampage.aspx?file=whaterver.pdf  you can then fiddle with the url and do mystreampage.aspx?file=..\web.config and with a little bit of trial and error navigate to the root directory and stream down the web.config  Over lunch I sat through a couple of GROK talks - JMeter  (load testing tool), Glimpse  (effectively like firebug for your server) and Out systems agile platform (the demo for this looked to good to be true I need to investigate further but in essence you model your business requirements and it spits out the code). WebSockets and SignalR by Chirs Alcock - This was very interesting deep dive into SignalR Event driven architectures by Ian Cooper - This was a very advanced session and alot of it went over my head however it was very well delivered and very interesting regarding SOA patterns. There was a session on Umbraco (Working effectively with Umbraco - Andy Gibbon) which Tony Southworth attended. Overall it was excellent, I even won something in the raffle at the end, a license for some software but no idea what!  I also won a t shirt although had to give it away as it was XXL and I am size S.  As we left we were all given a 1 month free subscription to Plural sight training website. After the event myself and the rest of the Preston Thai boxing massive went for an excellent meal at Mumtaz, Bradfords oldest and by far best curry house. All in all much fun and learning had by all. http://thecogworks.co.uk.aspx/blog/posts/2012/october/dddnorth2-quick-review/ http://thecogworks.co.uk.aspx/blog/posts/2012/october/dddnorth2-quick-review/ Mon, 15 October 2012 08:34:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-hackathon/ To take advantage of having some of the worlds brightest Umbraco talent in London all at the same time, we are organising a pre-festival hack day. Having spoken to the Umbraco HQ, we've decided that we should focus our minds on the live Umbraco bug list . Our main aim is productivity so we'd love to get as many issues closed off as possible during the day. So, to make sure we tackle the higher priority ones first, it would be great if we can get as much input as possible from the community. So, tell your friends, family, children, pets and anyone else you think would be interested to visit the bug list and vote. We will have, (hopefully) 20 crack Umbraconistas including Niels (the boss), Per Ploug, Lee Kelleher, Hendy Racher, Matt Brailsford and Anthony Dang in the room at the same time, so working collaboratively we should be able to get a fair amount done and make the day a great success.  Well, that and the fact the door will be locked and they won't be allowed to go to the pub until the whole list is finished :) So get along and vote and add any more to the list ASAP. Lets make Umbraco even better than it already is!! There are still some spaces left, so get yourself signed up if you're around the day before the festival. We'll keep you updated on our progress through the day as well. And don't forget kids, a Happy Hacker is a productive Umbraco hacker :) http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-hackathon/ http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-hackathon/ Wed, 10 October 2012 10:44:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-schedule/ The first stab at the Umbraco UK Festival talk schedule is now up on the website . We've tried to fit in as much as we can. The lineup is looking jam packed with useful information both about Umbraco and also general developer tools, hints and tips, so should be a good day allround for everyone. Highlights are:- The new Umbraco UI prototype Lucence and .NET Visual Studio 2012, TFS and other new developments Full automated Umbraco setup uComponents Load balancing and much, much more... You can see the full line up on the festival site . And don't forget the pre-festival Hack Day . Places are limited and filling up quickly, so get yourself on the list if you're around the day before. http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-schedule/ http://thecogworks.co.uk.aspx/blog/posts/2012/october/umbraco-uk-festival-schedule/ Thu, 04 October 2012 14:06:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2012/september/the-bespoke-agency-cms-is-deadlong-live-the-agency-cms!/ It still amazes me how many web agencies still use their own bespoke content management systems for client websites. A little history for you… In the beginning all we had was Notepad, Dreamweaver or dare I mention it, Microsoft Frontpage! We edited some text files, we FTP'd them with some images up to a webserver somewhere in the world and ta'dah we had a website. Then some clever clogs thought about adding a database into the mix. Built some code that worked some magic on the server to push the data to the web page, and then the 'dynamic' web was born. But, no that wasn't enough…we wanted more, our clients wanted more! They wanted control over the content. They started to ask 'Why do I have to keep phoning you up to make these changes? I want to do them myself, I know what I am doing now so make it so!!' and low and behold content management was born. The race to the top And so the race began…the period of time where every web development agency and software house in the world decided that they could build the perfect CMS - That piece of content management nirvana, their little piece of the web they could tie their clients into, charge them lots of cash for supporting, hosting, maintaining and developing to gain ongoing repeat revenue. And we here at The Cogworks are no strangers to this. 'NO!' we said. 'NO, NO, NO...We can build a better system than you!!' We know how the web works much better than all you other developers…mwaahahaha!!' However…What you don't realise at the time when you have that crazy idea, and you come up with that killer UI that you think will kill all other UI's and become the main way that the whole world will control the interweb, is 'Darn it, we need to support this thing. How the heck are we going to do that with just the 3 of us in a basement?' You have to field support calls like 'You what!! Why can't I paste from Word? What do you mean I have to paste it into Notepad first then paste it into your rich text editor!' or 'DAMN YOU [insert browser name] you've launched a new version that has just broken our beautiful UI. We've now got to rebuild the whole thing! How are we going to manage that while we're on the beach spending our clients hard earned cash on Pina Coladas.' Now, some companies made it a great success turning into major 'Vendors' i.e. Sitecore, episerver, tridion etc (you know them all by now) and some went on the hippy free thinking 'open source' journey i.e. Drupal, Wordpress and our own beloved Umbraco. End of the story…? At The Cogworks, we saw the light way back in 2008. We were about to embark on that painful journey and throw some of our children's inheritance at our very own cogCMS. But just before we did I had a quick look around, stumbled across Umbraco. Pretty much overnight we realised the error of our ways and dumped CogCMS in the bin with all the other failed systems, and specialised in Umbraco. Job done! We'd found an already fairly mature product, with a large community of devoted developers helping support, bug fix, maintain and improve the system. Cool. Surely thats it, everyone will have done the same...now clients won't be tied in, we won't have to support something forever with all the headaches that brings...End of story… Or so I thought... The Bespoke Agency CMS's Lives Over the past couple of years, and even now (well today in fact!) we're still coming across agencies still touting their own CMS's and clients are the ones ultimately suffering. They become tied into systems that basically just don't work and will never have the chance to keep up with the times. What started off as probably a very good idea, quickly falls into trouble when the client wants that new piece of functionality, or the browser version is upgraded and the UI is rendered useless. The agency simply does not have the resource to keep up, keep pushing the platform forward and keep innovating. They're never going to have the same chance that a large vendor with a huge amount of VC funding or an open source platform like Umbraco that has thousands of developers around the world working 24 hours a day to bug fix, upgrade and constantly create innovative new features. So please, clients beware of any agency that comes to you offering their little piece of the CMS world. Take a step back and think. What is going to happen in 6 months/18 months/2 years time when you've maybe based your whole infrastructure on this system and the agency goes out of business and no other developer in the world has the knowledge to support it. I know what I'd be doing...wheres that waste bin again?? http://thecogworks.co.uk.aspx/blog/posts/2012/september/the-bespoke-agency-cms-is-deadlong-live-the-agency-cms!/ http://thecogworks.co.uk.aspx/blog/posts/2012/september/the-bespoke-agency-cms-is-deadlong-live-the-agency-cms!/ Wed, 26 September 2012 16:02:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2012/september/media-indexer-for-examine/ Although released a while back thought I would blog about the CogUmbracoExamineMediaIndexer apologies for the name like all true developers we are rubbish at naming anything!   Anyhow the package is an extension for Examine, it's  a custom indexer to index any umbraco MEDIA node. Under the hood it makes use of apache tika ( http://tika.apache.org/ ) to extract content and meta data from umbraco media files. Tika can handle the following formats  http://tika.apache.org/ 1.2/formats.html#Audio_formats You can download the package from the package site at our.umbraco.org Enjoy! http://thecogworks.co.uk.aspx/blog/posts/2012/september/media-indexer-for-examine/ http://thecogworks.co.uk.aspx/blog/posts/2012/september/media-indexer-for-examine/ Fri, 21 September 2012 09:44:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2012/september/umbraco-uk-festival-update/ With only a month to go, the Umbraco UK Festival organisation is reaching fever pitch here in Chez Cog. Well as much excitement as 8 blokes in a room can muster. We've nodded some appreciation and grunted at each other a bit, so I think all things must be going well :) Things are in full swing and tickets are nearly sold out! The venue is booked (phew!), speakers are being lined up, the helicopters been arranged for Daniel Craig to make his entrance as Niels's stuntman, oh, and we have some great talks planned. The lineup so far:- Your favorite chief Unicorns Niels 'The Boss' Hartvig and Per 'Courier' Ploug Hansen will be presenting project 'Belle'. Umbraco community stalwart Lee 'uComponents' 'Kelleher presenting what's new and what you may not know about in the unrivalled uComponents. Umbraco veteran Ismail 'lucene' Mayat showing you how to get the best out of Umbraco Examine with some great hints and tips from a search guru. Tom 'SCRUM master' Smith preaching the ways of Agile best practice. Anders 'Teacommerce' Johansen launching V2 of the great eCommerce platform. Tim 'continuous deployment' Saunders introducing load balancing and deployment best practices And many, many more... Now we just need to add an extra 8 hours into the day somehow to squeeze them all in :) And don't forget the pre-festival Hackathon organised by Mr uBlogsy himself, our very own Anthony Dang. http://our.umbraco.org/events/pre-uk-festival-hack-day http://thecogworks.co.uk.aspx/blog/posts/2012/september/umbraco-uk-festival-update/ http://thecogworks.co.uk.aspx/blog/posts/2012/september/umbraco-uk-festival-update/ Thu, 20 September 2012 11:34:00 Tom Smith http://thecogworks.co.uk.aspx/blog/posts/2012/september/seo-friendly-redirects-with-url-rewriting/ Every time I launch a new site, I often find myself setting up two types of redirect. The first is to redirect the www version of the domain to the non www version of the domain (or vice verca). E.g. redirect www.thecogworks.co.uk to thecogworks.co.uk. Some domain hosts let you do this directly, alternaitvely you can do it in IIS on the webserver. The IIS method involves setting up another website, called something like "Cogworks Redirects", and then using IIS's built in HTTP redirect functionality to re route traffic. Unlike adding both domains as bindings on the website, this can return the all important 301 status code, and will therefore ensure that search engines only index content on the desired domain. This is a little annoying, as IIS on a particular server can become quickly clogged with redirects instances. I prefer to add all the bindings onto the website, and then use Umbraco's built in implementation of URL Rewriting .net to handle the redirection. This involves adding entries to the urlrewriting.config file which can be found in the config folder.  A 301 redirect from www to non www looks like the following: (Sorry it's an image - razor appears to be having issues escaping text containing the virtual URL syntax) Where virtualUrl corresponds to the location to redirect from, and destinationUrl, the location to redirect to. Changing the redirect mode corresponds to the type of redirect (and therefore status code) being retuned.  This can also be applied to the second type of redirect I often need to set up; links on the previous version of the site that have been indexed by search engines, but do not exist on the same URL on the new site. These links need to be cleaned up by setting up 301 redirects, which will cause the search engines to update their indexes to the pages' new locations. There are a number of excellent umbraco packages that allow content editors to do this on a page by page basis, however I sometimes find it quicker to do it manually. I tend to ask Google Webmaster tools for a list of 404s and then import these into URL rewriting.config. Redirecting a dead link from its old location to its new location looks like the following Differences include the redirect parameter being set to application, rather than domain, and regex matching being used on the virtualUrl and destinationUrl (the redirects are therefore domain independant). This regex matching feature is a powerful tool as it can also allow multiple pages to be cleaned up with only one rewrite rule.  Hopefully, this might come in handy when launching new versions of sites :)  Tom http://thecogworks.co.uk.aspx/blog/posts/2012/september/seo-friendly-redirects-with-url-rewriting/ http://thecogworks.co.uk.aspx/blog/posts/2012/september/seo-friendly-redirects-with-url-rewriting/ Thu, 20 September 2012 10:40:00 Tim Saunders http://thecogworks.co.uk.aspx/blog/posts/2012/april/membership-in-umbraco-51,-hiding-nodes/ I've been having a play with Umbraco 5.1 recently, particularly the memberhip functionality. One thing that had me stumped for a while was how to hide protected node or mark nodes as protected in a navigation. I had a trawl through the source of 5.1, which as every good knows is the best source of documentation.  Learn to read the source Luke... Umbraco 4 had the concept of IsProtected and HasAccess properties, so as you iterated through the ndes you could easily hide nodes that a user didn't have access to or mark them as protected. I used these in the Cogworks flexible navigation package in order to provide security trimming (hiding pages from the nav that the user didn't have acess to). Things have changed slightly in 5, to supress nodes in the navigation a member doesn't have access to you need to do the following: @inherits RenderViewPage@using Umbraco.Cms.Web; @{ var Homepage = @DynamicModel; while (Homepage.ContentType.Alias != "homePage") { Homepage = Homepage.Parent; } } <ul> <li><a href="@Homepage.Url">Home</a></li> @foreach (var item in Homepage.Children.WhereCanAccess()) { if (@item.CurrentTemplate != null) { var childName = item.Name ?? "(No name yet)"; <li><a href="@item.Url">@childName </a></li> } } </ul> The important bit in the above code is WhereCanAccess(), this returns a filtered list of nodes that are only visible to the current logged in user. So if you're not logged in or don't have permissions they will not appear. To mark a node as protected in the navigation you need to do this, well this is the only way I could find to do it (if there's a better way let me know!): @inherits RenderViewPage@using Umbraco.Cms.Web; @{ var Homepage = @DynamicModel; while (Homepage.ContentType.Alias != "homePage") { Homepage = Homepage.Parent; } var appContext = DependencyResolver.Current.GetService<IUmbracoApplicationContext>(); } <ul> <li><a href="@Homepage.Url">Home</a></li> @foreach (var item in Homepage.Children) { var isProtected = appContext.Security.PublicAccess.IsProtected(item.Id); if (@item.CurrentTemplate != null) { var childName = item.Name ?? "(No name yet)"; childName = (isProtected) ? "* " + childName : childName; <li><a href="@item.Url">@childName </a></li> } } </ul> Here we use the PublicAccess service to check if an individual node is protected and display an asterix (*) next to it if it is. So there we go, our first steps into Umbraco v5 Membership T P.S. Apologies for the dodgy code formatting :-) http://thecogworks.co.uk.aspx/blog/posts/2012/april/membership-in-umbraco-51,-hiding-nodes/ http://thecogworks.co.uk.aspx/blog/posts/2012/april/membership-in-umbraco-51,-hiding-nodes/ Wed, 18 April 2012 12:00:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2012/january/google-refine-with-cmsimport/ A happy new year to one and all, my first blog post of 2012 - hopefully more to follow projects permitting. I have used the excellent CMSImport on numerous projects both with SQL server and CSV data sources. &;In many instances the data is not always in the correct format and needs to be massaged. &;If using CSV as the data source typically you would use Excel or some simple text editor with a bit of search and replace, however there is a much better and powerful free alternative, enter Google refine . Check out the videos especially the third video on data augmentation . &;I followed the instructions and managed to insert latitude and longitude via Yahoo place finder into a CSV file that contained a list of retailers. You could easily write your own webservice that does some data transformation and provided the webservice returns json you can then use it to massage your csv data before importing using CMSImport. http://thecogworks.co.uk.aspx/blog/posts/2012/january/google-refine-with-cmsimport/ http://thecogworks.co.uk.aspx/blog/posts/2012/january/google-refine-with-cmsimport/ Tue, 03 January 2012 11:44:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2011/november/mid-range-net-cms-licence-fee-shocker!/ Well, not really a shocker...more a reality check. This is in reference to a recent article published on CMSWire about mid market web CMS vendors raising prices. Having used a number of content management systems over the last 10 years including Reef (ironically this 'sank' without a trace), Microsoft Content Management Server, Obtree, Immediacy (gulp) and Sharepoint (if you can call this a CMS) and sat through a number of other platform demonstrations such as Sitecore and episerver. I am aware of the weird and wonderful world of vendor pricing models. Some are per server, per domain name, per developer seat, per licence, or a combination of all these. Then there are the added (some hidden) extras such as installation fees, enforced training courses for developers, annual maintenance which is usually anywhere between 15% and 20% and of course the cost of actually implementing the system. The Shocker But, just how high some of the licence fees can escalate has actually surprised...no...shocked me! "...more than 3 content managers, you must purchase Sitecore's Professional Edition (includes Web CMS and OMS), which starts at US$ 60k (£40K) for the first server...you are more likely going to want a second server to support higher levels of traffic or internal development environments, etc. This means additional costs of anywhere between US$ 20 (£13k) and US$ 40k (£25k)" So in summary you are talking a figure approaching the $100,000 (£65,000) mark. And I'm sure this could quite quickly escalate in some circumstances. This is without any annual support, development fees, consultancy, hosting or integration with 3rd party applications. Lets just take a second and look at that number again...that's £65,000 (sixty five thousand pounds), or in real terms 65,656 things from your local 99p shop, admittedly you'd need a bigger trolley, 82,278 downloads from iTunes (UK) or 650,000 1p sweets, I like rhubarb and custard if anyone's feeling generous :) In terms of a full project, I'd expect the total cost to be nearly double the licence fees and then some, so you'd be looking at a total project cost of somewhere in the region of £130k - £150k by the time you'd tweaked, toned, perfected, mended and launched. This sounds like a huge sum of money, and don't forget those extras, there would also be the ongoing annual vendor support costs, which, if we take an average of 18% of the licence fee, would be in the region of £12k per year to cover upgrades and bug fixes, but this would not include the partner fee for installing the upgrades or further development work. So Why? My biggest question is why do companies feel they need to spend these sorts of fees on content management systems? Is it the number of extra features provided? Is it the perceived quality of the codebase? Is it the feeling of security spending a lot of cash gives? Or is it the fact that the decision maker feels they have someone else to kick if it all goes wrong? And don't forget, the systems highlighted in the article are just 'Mid Range' systems. I can't even begin to imagine the fees involved in the projects for the Vignette's of the world. The Open Source alternative...well Umbraco of course! There is an alternative out there...one that doesn't cost the earth and is as robust and secure as any commercial system. The alternative is the world of open source. I guess the historical perception of open source products is that they were written by Metallica t-shirt wearing, non-speaking, zero personality geeks in a dark room somewhere with no control or credibility (I gave my t-shirt away when my hair fell out by the way!). This is obviously not the case, given the success of systems such as Wordpress, Drupal and Umbraco. Taking Umbraco as an example, because this is what I know best, the reality is that there is as much control and structure as with any commercial offering, sometimes more. Umbraco is a commercial operation funded by support contracts, gold partnerships and umbraco.tv subscriptions. It now has 11 full time employees who are spread around the world covering every time zone. HQ's purpose is solely to provide stability, longevity and direction to the product. They don't provide professional services and don't have licences to sell to meet Board or shareholder targets Underneath the HQ is a team of devoted MVP's and core contributors who work tirelessly to bug fix and improve the platform, then there is the community as a whole, which must now number some tens of thousands of people around the world. Cost And how much does all this cost I hear you ask? Well the simple answer is simple. Nothing...zero...zilch... You still have to actually implement the system and integrate it into an organisation, but there is no more work than any other commercial offering out there in the marketplace, and it will be significantly more cost effective. In my experience, products that on the surface may seem nice and shiny, slick and well written with lots of modules that promise to magically solve all your organisation problems, still need an awful lot of configuration and set-up to get them to work just as you require. Sometimes more!!! There is never a catch all solution to any requirement. Every company has there own processes and needs and it is my opinion that a system should help solve and reduce the workload of the staff who use them. It should not add another level of complexity into an already busy day with yet another system you have to learn and get used to with its own nuances and quirks. Give it a go Anyway, I'm off to sit down and drink some sweet tea to get over the shock. In the meantime why not keep you wallet in your pocket for now and take a look at the open source world, preferably Umbraco of course and give it a go or even better come and see me and I'll show you. And if you do feel the urge to spend upwards of £50k on something, you could always give it to a much worthier cause :) p.s. these are obviously just my ramblings, please let me know if I am in any way innaccurate :) http://thecogworks.co.uk.aspx/blog/posts/2011/november/mid-range-net-cms-licence-fee-shocker!/ http://thecogworks.co.uk.aspx/blog/posts/2011/november/mid-range-net-cms-licence-fee-shocker!/ Fri, 11 November 2011 11:16:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2011/february/umbraco-manchester-meetup/ Following hot on the trail of the regular London meetup and first ever Preston meetup we present the first Umbraco Manchester meetup. &;Hopefully will be a regular monthly event. See Manchester meetup home page for more information. You can also follow us on twitter @ManUmbMeetup. http://thecogworks.co.uk.aspx/blog/posts/2011/february/umbraco-manchester-meetup/ http://thecogworks.co.uk.aspx/blog/posts/2011/february/umbraco-manchester-meetup/ Mon, 21 February 2011 11:54:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2011/february/dataset-to-xpath/ Just a quick tip for all you xslt cowboys. &;I am currently working on an existing Umbraco installation. This site utilises webservices that return datasets. &;The display of data has been done using .net usercontrols that make calls to the webservices, unfortunately for me alot of the layout has been done with codebehinds and stringbuilder. The client wanted new subsites with different layout, so rather than hacking around with the existing usercontrols I decided to use xslt macros instead this gives me greater flexilbity to customise layout without having to also rebuild and deploy dlls. As all you umbraco gurus know to use custom third party data in umbraco xslts you need to write xslt extensions and provided your method returns XPathNodeIterator it will all work. So at first I was going to just create a method that got the dataset from the webservice then read that into and xml document then return XPathNodeIterator from the xml document. &;However I did a quick google and found DataSetNavigator so instead I created an extension method that looks like public static XPathNodeIterator CreateNavigatorOnDataSet(this DataSet ds) { var nav = new DataSetNavigator(ds);                   return nav.CreateNavigator().Select("/"); } and my xslt extension method looks like public static XPathNodeIterator GetPracticePeopleXml() { DataSet people = WebHelper.GetPracticePeople(); return people.CreateNavigatorOnDataSet(); }   http://thecogworks.co.uk.aspx/blog/posts/2011/february/dataset-to-xpath/ http://thecogworks.co.uk.aspx/blog/posts/2011/february/dataset-to-xpath/ Fri, 18 February 2011 09:38:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2011/february/preston-meetup/ So last night was our inaugural Preston Umbraco meetup held at the excellent Indian restaurant East is East. &;Considering Preston is a bit of a backwater compared to that London we had a good turn out. The swag (we still have loads of it knocking around in the office) and curry went down very well. Topics ranged from Umbraco to .net in general, there was a running joke throughout the evening that Phil was some kind of Sharepoint expert you had to be there to get it! The really strange thing for me last night was the amount of six degrees of separation we all seemed to have worked with or met someone already known by the others. Apologies for the picture quality think the waiter needs some photography lessons from our resident Umbraco photographer Mr Doug Robar! &; (Left to right: Chris Gaskell - HeapsAndStacks Ltd, Phil Winstanley - Microsoft, Ismail Mayat - The Cogworks, Tony Southworth - Delineo, Neil Gibbons - The Building Blocks) We are hoping to arrange another meetup some time next month but this time in Manchester hopefully that will draw a larger crowd. &;If we can get something regular going then maybe we can look at venue where we can get screens and some wifi so people can plug in and do some demos etc. Watch this space or keep an eye out on the events section of our.umbraco.org http://thecogworks.co.uk.aspx/blog/posts/2011/february/preston-meetup/ http://thecogworks.co.uk.aspx/blog/posts/2011/february/preston-meetup/ Fri, 11 February 2011 09:04:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2011/february/essential-listening/ Over the last couple of months I have been regularly listening to the excellent podcast This developers life by Rob Conery (He of Subsonic and Tekpub fame) and Scott Hanselman (Microsoft Senior Program Manager). The podcasts cover a wide range of topics and you get to hear from leading industry gurus and wizards, plenty of funny stories and insights. &;Its free and also available from iTunes. If your in the IT game its a must!! http://thecogworks.co.uk.aspx/blog/posts/2011/february/essential-listening/ http://thecogworks.co.uk.aspx/blog/posts/2011/february/essential-listening/ Mon, 07 February 2011 09:05:00 Tim Saunders http://thecogworks.co.uk.aspx/blog/posts/2011/february/where-did-my-content-nodes-go/ I had what can only be described as a mouse induced panic attack yesterday. I was adding content and moving nodes about in the Umbraco editor with reckless abandon, when pow all of a sudden all my nodes disappeared! EEEK! I navigated away from the content tree into another section in the hope that they were just shy and if I stopped looking at them they may re-emerge. Nope, I returned to the content section and they were still gone (Sort of the opposite of Schrodinger's cat). Then I had one of those rare moments of clarity and realised I had accidentally double clicked the root content node. A quick double click of the content node and all was well again. Just one of those gotchas that crops up from time to time. I put it down to an excess of coffee and over excitement at the speed of the new Umbraco content tree. http://thecogworks.co.uk.aspx/blog/posts/2011/february/where-did-my-content-nodes-go/ http://thecogworks.co.uk.aspx/blog/posts/2011/february/where-did-my-content-nodes-go/ Sat, 05 February 2011 13:19:00 Tom Smith http://thecogworks.co.uk.aspx/blog/posts/2011/january/n-word-preview-for-blog4umbraco/ The CogBlog, our staple bastion of informative comment (at least some of the time!), is powered by the Blog4Umbraco package. There are a couple of additional modifications that we've made here and there to get it running just how we like it. These modifications were a chance to look more deeply into xslt extentions and showcase just how easy it is to implement them; all thanks to the power of Umbraco. Blog4Umbraco Fist thing's first; as a package, Blog4Umbraco (b4u) isn't updated too frequently. Additionally; there are several versions out there which produce different levels of stability depending on which version of Umbraco you're running. However, we've been very pleased with the results we've obtained after a little bit of tweaking. For anything pre Umbraco 4.5.2, the latest release version of the b4u package (available from codeplex here ) should be fine (version 2.0.26 at the time of writing). Umbraco 4.5.2 implements a new version of the XML Scheema; as a result a different version of b4u is needed;  Blog4Umbraco 2.0.26 for 4.5.x is available here . At the time of writing some XSLT issues are still being reported, so you may wish to download the latest version of the b4u source (available here ) and have a look at the try-codegeko-features branch where Benjamin Howarth over at Codegeko has ironed out a number of these issues. The problem; displaying n words of a blog post, with intact HTML So; assuming you have a working install of b4u, you may have noticed that any blog posts will be displayed in full on the blog homepage. This looks fine if using the bundled blog skin, however, we wanted our blog to look like a content page within our existing site structure. Once we had applied all our templates and styles, we ended up with an extremely long blog homepage, where each post was too long and thin to be legible, and finding posts was a bit of a nightmare. Other blog packages, such as wordpress, automatically only display a 50 or so word summary on the blog homepage. This makes the page much more navigable, so we began looking for a way to allow b4u to do the same. After some extensive googling, it appeared that most of the solutions involved using the umbraco.library:StripHtml() method to remove any html from the post and then using the xslt substring method to print out x characters from the post. This didn't really satisfy us as we wanted to make sure that HTML was included, and that it would be well formed. You could not simply strip out the html and print out the x characters as; if the last character in the post substring was in the middle of an open HTML tag, you would end up with page errors as the tag would never be closed. The solution; HTML agility pack and an xslt extension In order to make sure that we could print out our n words and still maintain well formed html, we made use of HTML agility pack . In the creator's own words, it is "a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" malformed HTML. The object model is very similar to what proposes System.Xml, but for HTML documents (or streams)." as a result, it was perfect for our needs. There is also a version included with the Umbraco core; so you may not even need to upgrade your DLL. However, we included the latest version of the library to be on the safe side. The next step was to write an xslt extention; this would provide a getSummary method to be used within b4u's BlogListPosts.xslt. As with the Cogworks DayNight Webservice , Umbraco makes it easy to integrate your code libraries into the core. I won't explain how the extension itself uses HTML agility pack here, as I have a bit of a thing for excessively commented code, which does all the explaining for me! You can download the C# class below; Download the C# code here! Wiring the extension up To use the extension, you first need to build the code file above and drop the resulting DLL into the bin directory of your website. (Make sure you also have the HTML agility pack DLL in your bin - remember; you may need to upgrade this DLL to the latest version as mentioned above). You then need to add a reference to it in your xsltExtentions.config file like this; <ext assembly="BaseExtentions" type="BaseExtentions.stringMethods" alias="CogParser" /> In the desired xslt file, in this case b4u's BlogListPosts.xslt, you need to add the extention's namespace. In order to use it like an xslt method, you then need to add the extension to the exclude result prefixes list. The stylesheet tag should thus look something like this; <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxml="urn:schemas-microsoft-com:xslt" xmlns:umbraco.library="urn:umbraco.library" xmlns:tagsLib="urn:tagsLib" xmlns:Exslt.ExsltStrings="urn:Exslt.ExsltStrings" xmlns:BlogLibrary="urn:BlogLibrary" xmlns:SmithParser ="urn:CogParser" exclude-result-prefixes="msxml umbraco.library tagsLib Exslt.ExsltStrings BlogLibrary CogParser"> The extension can then be used like a regular xslt method; like this; <div class="entryContent"> <xsl:value-of select="CogParser:GetSummary($post/bodyText, 100, umbraco.library:NiceUrl($post/@id))" disable-output-escaping="yes" /> </div> Where $post/bodyText is the HTML blog post being worked on, 100 is the number of words you want back, and the last argument is the link to the document to be put out as the read more link. We also use this to generate blog post previews for our blog RSS feed. So there you have it! An n word blog post preview, with valid HTML. All thanks to the power of xslt extentions and Umbraco! Issues / Extending You might run into an issue with the HTML space character. If you add the following entity to the doctype tag in the xslt you shouldn't have any problems; <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE xsl:stylesheet [ <!ENTITY nbsp " "> ]> Additionally, the handling of the read more link is hard coded in the extension so that it can appear within a blog post's p tags. (i.e. on the same line as the last part of the n word preview). You may wish to adjust this behaviour to suit. http://thecogworks.co.uk.aspx/blog/posts/2011/january/n-word-preview-for-blog4umbraco/ http://thecogworks.co.uk.aspx/blog/posts/2011/january/n-word-preview-for-blog4umbraco/ Fri, 21 January 2011 12:41:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2011/january/umbraco-gold-partner-what-it-means-to-us/ It was the end of November 2010, and the days were cold and dark, work was piling up and we were about to be buried under one of the worst snow storms for years. Well what could bring us out of this gloom and despair... Well, A nice shiny email from the founder of Umbraco, Niels Hartvig...all glistening in the cold morning gloom...and what, you may ask, was the email about? The title certainly woke us up...'Partner News: Introducing the Gold Partner Program'.&; It got us interested :) Umbraco Gold Partner I here you say, what's that all about then? Well admittedly, it sounds a bit 'blingy' (if you're not from the UK, then this may not mean anything, so think Cristal Champagne, 50 Cent, BA Baracus and Goldfinger rolled into one) This isn't to say that everyone at the CogWorks has started wearing open shirts and Gold medallions.&; Far from it.&; We are all very reserved I think you'll agree :) But it does mean we can now offer a range of features, services and support to anyone and everyone who would like to use Umbraco as their chosen CMS, and lets face it, why wouldn't you? The Gold Partner program includes access to the Umbraco Complete package including Courier, Concierge and Contour and a guaranteed 24 hour response time allowing us to escalate issues to the core team. However not only can we offer these excellent features, but we are now, with our licence fee, helping to make Umbraco better for everybody who uses it. The fee goes directly to paying for development and bug fixing of the software itself, so whilst we are always looking to improve Umbraco with our own code submits to the source, we are now directly supporting development of the software itself. So not only do we now create and host the Umbraco UK Festival , the regular monthly Umbraco London Meetup , organised and hosted the Umbraco 5th Birthday celebrations and created the new Umbraco installer process , we now are directly funding development of the product. Not a bad demonstration of our commitment to Umbraco and our aim to become the number 1 provider of Umbraco design, development, integration and other related services in the UK and beyond. So as you can see...even the darkest, coldest days have a nice goldy shimmery light at the end of the tunnel :) http://thecogworks.co.uk.aspx/blog/posts/2011/january/umbraco-gold-partner-what-it-means-to-us/ http://thecogworks.co.uk.aspx/blog/posts/2011/january/umbraco-gold-partner-what-it-means-to-us/ Mon, 17 January 2011 12:09:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2011/january/the-new-umbraco-installer-process/ As announced at the UK Umbraco Festival by Niels Hartvig, the Founder of Umbraco, we have been asked to help design the new Umbraco installer process to be released with Umbraco 4.6 (Juno). So I thought is was about time I wrote something about what has been going on. Why does Umbraco need a better installer? Historically the installer has been a fairly manual process aimed at the developer community.&; This was until Microsoft added Umbraco to their Web Platform Installer . This added an element of automation to the process by providing all the prerequisites in one place. However, the process was still a fairly plain, dry experience only providing a blank Umbraco installation with no real instructions as to what to do next for the new Umbraco user. The user was left with a plain white 'Welcome to Umbraco page' with no real DocTypes, templates or content installed.&; There was no example of where to start or what Umbraco can do. So unless you were familiar with how Umbraco worked or indeed Microsoft.NET master pages, and the basics of CMS design and development, then there was quite a steep learning curve to overcome.&; Believe me, we had to climb it 3 years ago :) The Dream Solution So, Niels's dream was to provide a much more user friendly installation experience with some level of interactivity and a means for the first-time user to see how all the separate areas of Umbraco fit together. And so the new, shiny Umbraco installation process was born... After some great initial work by Tim Geyssens and Per Ploug Hansen to create the installation process and the skinning functionality, it was down to Dan Ashton, Creative Director at The CogWorks, to wave his creative magic wand and take a non-technical view of the process, both from a user experience and a design perspective. What has been created is something that, I'm sure you'll agree is both visually appealing and easy to use. The process now flows from step to step taking the user through the necessary stages, giving the option to install a starter package or if they are a more advanced user, to run with the standard basic blank install. There are 4 starter kits each providing different elements to get you up and running in no time at all, and also the ability to be able to skin the kits with one of a number of great new designs. The plan is to add more designs skins on a regular basis. So all in all a great result for Umbraco and The CogWorks...long may the partnership exist in happy designery heaven...with a sprinkling of technical cheer :) http://thecogworks.co.uk.aspx/blog/posts/2011/january/the-new-umbraco-installer-process/ http://thecogworks.co.uk.aspx/blog/posts/2011/january/the-new-umbraco-installer-process/ Sun, 09 January 2011 20:17:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2011/january/similarity-new-ucomponent-datatype/ Over the Christmas period while in between things I have been working on a new datatype for the excellent uComponents , I have named the datatype Similarity. Similarity is another content picker. The question arises we already have plenty of content pickers why do we need another? Well all the content picker pickers to date work on the principle of strong linking namely user picks content or media from the tree an item they would like to link to current content. &;Similarity works on the principle of conceptual linking. In conceptual linking the content of the current document is used to find other content that is like the current document. &;The datatype type works in the following way: You configure the datatype to point to a Lucene index (must be content index not pdf). Then you set the fields that you wish to compare (Can select more than one field however only fields of type textstring, text multiple and rich text editor are presented for selection). Add the datatype to a document type. When the type renders you click on find similar and a Lucene more like this search is performed. The query returns documents that are similar to the current one using the fields specified in the datatype settings. &;You can then select and sort the suggested documents. The datatype will only really work with a large content rich site good examples of content rich sites are public sector websites. There are a few things that I need to do with this datatype including adding an xslt extension method so that you can use the more like this query in your xslts. I am hoping Similarity will be part of the next release of uComponents. http://thecogworks.co.uk.aspx/blog/posts/2011/january/similarity-new-ucomponent-datatype/ http://thecogworks.co.uk.aspx/blog/posts/2011/january/similarity-new-ucomponent-datatype/ Tue, 04 January 2011 08:24:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2010/december/examine-index-admin-package/ Just released a new package on our.umbraco.org called Examine index admin. Its a simple dashboard control that lists all your examine indexes and allows you to rebuild them. Hopefully when I get time I will add new features. Get it here if you like it dont forget to vote up. http://thecogworks.co.uk.aspx/blog/posts/2010/december/examine-index-admin-package/ http://thecogworks.co.uk.aspx/blog/posts/2010/december/examine-index-admin-package/ Fri, 17 December 2010 16:49:00 Tom Smith http://thecogworks.co.uk.aspx/blog/posts/2010/december/snow,-festive-fun-and-the-cogworks-site-banners/ Here at the Easter Egg department, we've been busy cooking up some seasonal decoration for the Cogworks site, whilst having an educational (and therefore entirely justifiable to HR...) look at some of the features of CSS3, jQuery and Umbraco Base. The Day / Night Banner (Webservice using Umbraco Base) You may have noticed that our homepage banner currently has two "themes"; a day scene and a night scene. When you visit the site, you are geo-located by IP address, and your latitude and longitude are used to find the sunrise and sunset time in your location. If it's after dark where you are, then you get the night version of the site, otherwise you see the day version. This is all down to our DayNight webservice which was built using the awesome power of Umbraco Base. As the site homepage loads, the webservice is called using jQuery. $(document).ready(function () { is used to make sure that that the webservice is only called after the main page elements have loaded. All the heavy lifting involved in turning a standard, DLL compiled C# class library into an umbraco base webservice is handled by the umbraco core itself. Simply declare a namespace, in this case "BaseExtentions", create a public class, in this case "DayNight", and a public static method, in this case "Get". Build the project, drop the DLL in the website's bin folder, and amend your restExtentions.config file as follows; <ext assembly="/BaseExtentions" type="BaseExtentions.DayNight" alias="DayNight"> <permission method="Get" allowAll="true" /> </ext> Umbraco works it's magic and gives you a fully functioning webservice at http://thedomain.com/Base/Alias/Method/Param1/Param2/.../ParamN without any more fuss. In our case, /Base/DayNight/Get/...parameter list. Credits: Props to freegoip.net which we use for geolocating by IP and earthtools.org which we use for determining sunrise and sunset times Snow!!!! (jQuery and CSS3) The festive period has come and so we decided to add some seasonal cheer to the site! Dan, leader of our fantastic design team, created the winter scene currently shown on the site homepage during the day. After one of those fabled "wouldn't it be great if..." conversations, the topic of snow came up and was duly referred to the Easter Egg department for immediate action. The banner system described above uses an Umbraco Macro and associated .net usercontrol. This injects the jQuery responsible for calling the DayNight webservice into the head of the homepage, and registers it as a startup script. As a result the groundwork for implementing somthinghitme's jQuery snowfall plugin was already done. The plugin works by creating a matrix of <div> elements of the same base CSS class. It accepts a few options such as max /min flake size, speed etc, and uses random number functions to adjust each individual flake's markup. This gives you a nice, natural, random feel, where no two flakes are too alike, resulting in a superior snow effect. Animation wise, the matrix is then iterated over in a manner not too dissimilar to game matrices. This gives you an effect as below; This looks good, but it doesn't take long to notice that the flakes are square; they still look a bit too true to their original nature as DIV's with #ffffff backgrounds. As a result, I decided to extend the plugin to allow a snowflake image to be used. Of course there are many ways to do this that would avoid using CSS3. You could give each flake an internal <img> element with the same width and height as the flake itself and set the background to transparent. However, the larger amount of markup falling down the page seemed to reduce the performance somewhat! I also wanted to try out a CSS3 solution. CSS3 includes the background-size property which allows you to scale a background image to fit an element. As a result, in CSS3 enabled browsers (Chrome, Safari, etc) you should see flakes like this, The problem was that this looked pretty poor in non-css3 enabled browsers. As a result, it was necessary to detect if the client's browser supported CSS3. I did this using Jeffrey Way's CSS3 detection script ; This creates a test element with the desired CSS3 property, and then looks to see if that property remains in the final markup. It returns a boolean and is thus nice and simple to incorporate into your code. As a result, CSS3 enabled browsers get flakes and all other browsers get blocks. The iPhone's version of Safari is CSS3 enabled and handles the animation smoothly enough, thus the performance test is passed. I also hooked it into our homepage doctype so it can be turned on an off at will from the Umbraco back end. End result; festive cheer, all thanks to jQuery, CSS3 and the power of Umbraco! http://thecogworks.co.uk.aspx/blog/posts/2010/december/snow,-festive-fun-and-the-cogworks-site-banners/ http://thecogworks.co.uk.aspx/blog/posts/2010/december/snow,-festive-fun-and-the-cogworks-site-banners/ Wed, 01 December 2010 14:21:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2010/december/are-you-making-use-of-microformats/ Using microformats within HTML code provides additional formatting and semantic data that applications can use. For example, applications such as web crawlers can collect data about on-line resources, or desktop applications such as e-mail clients or scheduling software can compile details. The use of microformats can also facilitate "mash ups" such as exporting all of the geographical locations on a web page into (for example) Google Maps to visualize them spatially. (Ref wikipedia ) We use it on our website wherever we display our address. &;In the backend on the home page node we have a repeatable custom content datatype and the xslt macro writes out that content as an hCard http://thecogworks.co.uk.aspx/blog/posts/2010/december/are-you-making-use-of-microformats/ http://thecogworks.co.uk.aspx/blog/posts/2010/december/are-you-making-use-of-microformats/ Wed, 01 December 2010 13:59:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2010/november/my-first-umbraco-452-site/ I know Umbraco 4.5.2 has been out for a while but only recently have I got round to building my first 4.5.2 site. The site is for a company called World of Incentives you can view the site here . Some highlights with the site build: Product and main site search created with UmbracoExamine, I made use of Examine action handlers to inject all content into one field and also to transform content picker data from node id's into actual values. On full text search result pages implemented Highlighter.net to highlight search text matches See fig 1 Product search form filters created using Linq2Umbraco Made use of awesome uComponents multi node tree picker to tag products with appropriate meta data e.g country, theme (created these values as nodes and pointed uComponent data type to that tree) See fig 2,fig 3 Created multi step form with Contour hack to enable progress step bar see request for proposal form also created contour custom workflows to handle emailing of request form submissions to Destination management companies (DMC's) Fig 1: Search results with word match highlight Fig 2: Multi node tree picker in action to set site global footer links Fig 3: Multi node tree picker used to pick product images Overall very impressed with 4.5.2 and cant wait for Juno!! http://thecogworks.co.uk.aspx/blog/posts/2010/november/my-first-umbraco-452-site/ http://thecogworks.co.uk.aspx/blog/posts/2010/november/my-first-umbraco-452-site/ Fri, 19 November 2010 09:51:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2010/november/umbraco-uk-festival-niels-hartvig-umbraco-founder/ Well, here's the 2nd blog post from the Umbraco London Festival 2010...enjoy! Niels Hartvig - Umbraco Founder The first talk was from Niels Hartvig, the founder of Umbraco who gave a great potted history of Umbraco, where it came from and where it is going. You can see Niels's talk on the Umbraco Streaming site . It's always interesting to listen to Niels talk so passionately about Umbraco and how it has grown from a small glint in his eye to where it is now, with a fast growing international community of users and developers, who seem to be equally as passionate about the product as he is. And it is this community of devoted Umbracians that gives Umbraco the edge over some other CMS's in the market today. It's what drives the product forward and makes it such an easy to use and flexible platform both for developers and content editors. Anyway, without waffling on too much, here are some salient points from Niels' speech and the relevant times in the video so you can jump to them if you wish...:- Umbraco history - 3m:00s Niels's description of where Umbraco started is a great talk and gives a great insight into the history of Umbraco and where it started. User interface core team - 24m:20s Niels honoured us with some fantastic announcements at the event. The first of which was the introduction of a second core team that will work in conjunction with the current core team. The new core team will focus on user experience and interface design of the product as opposed to technical areas or the Umbraco CMS. This is something we feel will add even greater value to the platform, ensuring any new developments and user interface research is integrated into the admin interface. New Shiny Umbraco.org site - 28m:10s Niels also announced the launch of the new and improved umbraco.org website which he made live at the event with the click of a mouse. Even over one of the slowest 3G networks in Europe Umbraco worked like clockwork and then it was live. Microsoft relationship - 33m:33s Microsoft have been particularly interested in Umbraco of late, Paul Sterling, one of the Umbraco HQ has been working closely with Microsoft in the US and also a lot more Microsoft evangelists are talking about Umbraco. MS SQL Compact Edition for Umbraco Juno v4.6 - 34m:06s To make Umbraco as flexible as possible, MS SQL Compact edition will be available as part of the installer process for Umbraco Juno making it able to be installed in Medium trust on the some of the more cost effective web hosts. Prodigy Windows Phone 7 plugin and API - 34m:32s Niels demonstrated a new webservice API and a reference application for windows phone 7 that he has been working to develop with Microsoft. It allows you to easily create and connect to mobile devices - Windows Phone 7 in particular. The API provides an easy way to create apps for WP7 to allow editors to monitor administrative tasks whilst on the move if required. New default dashboards controls - 45m:00s New default dashboards have been added to Umbraco Juno so more information can be added easily to the dashboard when a user first logs in. More options have been added such as 'Show Once' which will display a tickbox allowing the user to hide the control thenext time they log in. Also permissions have been added to dashboard controls to allow greater control over who see what. Installer screens and JUNO - 47m:22s The announcement that we at The CogWorks are particularly proud of was the demonstration of Juno (Umbraco 4.6) and the new simple installer and skinning process for users to get up and running quickly with Umbraco. The CogWorks are working with the Niels and the Umbraco core team to fully re-design the Umbraco installer process. It is a project that Dan Ashton our Creative Director has taken on personally, drawing on his years of design, information architecture and usability knowledge, to make the process as user friendly and easy to use as possible. We'll be posting some blogs about this design process and how its progressing over the next few weeks. Skinning enhancements 48m:13s Connected to the installer process is the addition of a range of default skins for Umbraco. As part of the installer process, the user is asked whether they wish to install a skin. They can then adjust colours, logos, banners etc. Module injections 51m:59s Also connected to this is the ability to add modules to the skin really easily Well, I think thats about all for Niels' talk. Let me know if I have missed anything or got anything wrong and I will add update the entry. http://thecogworks.co.uk.aspx/blog/posts/2010/november/umbraco-uk-festival-niels-hartvig-umbraco-founder/ http://thecogworks.co.uk.aspx/blog/posts/2010/november/umbraco-uk-festival-niels-hartvig-umbraco-founder/ Wed, 17 November 2010 00:30:00 Ismail Mayat http://thecogworks.co.uk.aspx/blog/posts/2010/november/migrating-to-umbraco-via-cms-import/ For those of you who have been following my twitter feed will have seen over the last few weeks some interesting tweets regarding migrating from Immediacy CMS to Umbraco . We had an old client who was running Immediacy 5.1, this CMS is no longer supported and client wanted to move to Umbraco but wanted to migrate existing content (html,media and members) over to Umbraco. Enter the fantastic CMSImport 1.1 created by Richard Soetman. This excellent package made the migration process a breeze. Out of the box it can migrate content and members from numerous sources. &;In this instance the source was a SQL server database the query below (updated p_parent_id for each section of the site, currently CMSImport cannot do hierarchical import) was used on each section of the old site to retrieve content to import The pro version enables import of any related media from imported content, you have to ensure that the images and files referenced are copied over to the root of your Umbraco installation in my case I had docs, pdf,images,system_images folders in Immediacy. I copied these over so that during the import CMS import could pick up and import those files/images. However one issue that cannot be handled out of the box and I guess is an issue with any migration tool is recreation of links between content pages. Just as Umbraco in the backend maintains content links using its own syntax namely {localLink:1234} Immediacy also does something similar i.e ILINK|1234 so although all the content can be migrated and existing links would still work as links are created using page names, in the Umbraco backend those links would be stored as /about-us/somepage.aspx. &;This would mean initially the links would work however if at any point the target page was renamed or moved then republished its url would be different and links would be broken. &;So how do we migrate over all internal content links? Thankfully CMSImport has a really nice feature called FieldAdaptors . It's an interface you implement and for a given datatype (in this case rich text field) you can before actual save of newly migrated content perform some processing. So I created my own field adaptor, for my document type that I was migrating to I created an extra temporary field called Immediacy Original Id. &;During the migration setup I mapped the Immediacy page id to this field. In my fieldadaptor I loaded the html content into an Htmlagility pack document then using xpath retrieved all anchor tags that contained file links (.doc,.pdf) in the href attribute. &;For all links that contained a file link I made use of method MediaImportHelper.ImportMedia which is part of CMSImport pro api. &;This imported the Media and fixed the link so that it pointed to a valid Umbraco media item. I then in the field adaptor processed list of all Immediacy links (basically got all links with ILINK| in the href attribute), using an Immediacy database stored procedure (immediacy_GetPagePath&;Immediacies equivalent of NiceUrl) I converted the ILINK to a full link e.g /about-us/somepage.aspx and I also tagged onto the link the following querystring ?immPageId=1234 where 1234 was the original Immediacy target page Id. The first migration pass gave me all content and Media imported. &;The content urls worked as page names were the same as Immediacy urls, however they were not proper Umbraco internal links. I then hacked the excellent content maintenance dashboard control written once again by Richard Soetman (no stopping this lad!) and added an extra button to it called fix Immediacy Fix Links. This when clicked for selected documents gets the bodyText field content extracts all links that contain ?immPageId=1234 and for the Immediacy page id makes an Umbracoexamine (after initial migration pass ensure the internal index is up to date, I did a manual rebuild) call into the internal index and gets the document with that original Immediacy value. &;From that it gets the Umbraco page id and updates the link from /about-us/somepage.aspx to {localLink:1234} thus all internal links are converted to Umbraco links. &;You could have done the umbraco document lookup using linq2umbraco or xml xpath however you would need to publish all content first, also it would not be as fast as Lucene. One side effect of this is that after the update of links and subsequent save and publish, the update date of documents is set to date of migration and original update date from Immediacy is lost (this effects ordering of news lists that are dependant on update date for sorting). &;However a second pass migration with just update date field mapped will fix this problem. After performing the migration I copied over the templates html and removed Immediacy plugins and replaced them with Umbraco macros &;e.g left navigation, sitemap list news the usual suspects that no doubt all of you have created during your Umbraco careers. The site had a couple of forms and I recreated those using Umbraco contour. One of the forms was registration form so I created a custom workflow to create the new member upon form submission. &;The old site had extranet facilities and using CMSImport I migrarted all 300 members over and setup the protected area via Umbraco public access. I also setup UmbracoExamine with PDF indexing so we now have a proper search as opposed to Immediacy default Index server we now have Lucene.net with search word highlighting. &;Still a few bits todo on the site but will post link to live site when its up. And that is how you migrate from Immediacy to Umbraco!! I would like to thank Richard Soetman for all his help on this project without CMSImport this would have been a big job. This method could in theory be used for other CMS migrations, if you can get to the CMS content (its in database or xml files or csv) and you have a way of accessing the link generation method used by the CMS then this would work. Please note updating content and migrating files/images can only be done with CMSImport 1.1 pro. http://thecogworks.co.uk.aspx/blog/posts/2010/november/migrating-to-umbraco-via-cms-import/ http://thecogworks.co.uk.aspx/blog/posts/2010/november/migrating-to-umbraco-via-cms-import/ Fri, 05 November 2010 11:56:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2010/november/the-umbraco-uk-festival-2010-where-it-came-from/ Well firstly...What a day!! Secondly many thanks to all those who made the effort to come, especially all our friends from further afield - i.e. Belgium, Holland, Denmark, Kent...well it was a bit of an effort for @munkimagik to get in on time :) If you couldn't make it or didn't manage to get a ticket, then I will be adding a series of blogs describing briefly what happened throughout the day, so keep an eye out, I'll tweet it around when I write them. And don't worry, because of the success of this one, the next one will be bigger and better and with added Bling!! Why a UK Umbraco Festival? It all started back in February really when myself and @warrenbuckley decided to put together the Umbraco birthday event which went down a storm.&; And I'm not exaggerating! We nearly literally did sink due to the rocking of the boat and the waves from the wind on the Thames. I'm still not sure if @will_coleman has recovered yet. It was nearly sick bags all round and abandon ship, which luckily didnt turn into a massive man fight for the lifeboats. The seeds were sown in my head at that point that I wanted to do a more regular larger UK Umbraco event to showcase Umbraco and how it is evolving as a product, hence the birth of the Umbraco UK Festival . Also I wanted to give something back to the community. Since we found Umbraco a couple of years ago now, we have found the community to be absolutely amazing and it's what sets Umbraco apart from other CMS's. I've been in the web industry for nearly 13 years now using various technologies and content management systems, and I have never been involved in such a friendly, helpful and active community, so this is my little piece to give back to everyone. The Event The event was only supposed to be a small get together of UK based Umbracians, but the response was absolutely amazing. After adding the event to the diary on our.umbraco.org spaces started flying out the door.&; And not only to UK based people.&; Morten Christensen - @sitereactor and Richard Soeteman - @rsoeteman added themselves to the event and so started the foreign attack.&; And when we found out that Niels Hartvig - @umbraco was going to come over from Denmark it added an extra buzz to the event. After selling out in 9 hours, we had 120 people apply for the tickets which Niels pointed out to me on the day would have made it the 2nd largest Umbraco event after Codegarden this year...wow!! Basically the day turned into a mini Codegarden, or Codegarden's little brother which was totally unplanned, but totally amazing to see and be involved in. So, thats the reasons...now onto the day itself.&; Part 2 coming soon... http://thecogworks.co.uk.aspx/blog/posts/2010/november/the-umbraco-uk-festival-2010-where-it-came-from/ http://thecogworks.co.uk.aspx/blog/posts/2010/november/the-umbraco-uk-festival-2010-where-it-came-from/ Wed, 03 November 2010 23:49:00 Adam Shallcross http://thecogworks.co.uk.aspx/blog/posts/2010/november/finally,-we-have-a-blog/ After years of talking about it, pondering subjects and general busyness we have finally managed to get a blog up and running on our website. Now we have one, we have the mammoth task of trying to keep it up-to-date and full of intersting and inspiring articles. Easier said than done I am sure. Anyway, we will try and pass on useful chunks of information on a number of different subjects from design, though Umbraco hints and tips and general worldly observations on life, the webiverse and anything else that crosses our minds on a day-to-day basis. Please let us know if you have comments on our ramblings and thanks for taking the time to read and ponder with us :) Cheers Adam http://thecogworks.co.uk.aspx/blog/posts/2010/november/finally,-we-have-a-blog/ http://thecogworks.co.uk.aspx/blog/posts/2010/november/finally,-we-have-a-blog/ Wed, 03 November 2010 23:37:00