Miguel Paraz

I just finished reading Application Testing with Capybara for a review requested by the publisher, Packt Publishing.

I know actually know very little about Ruby on Rails, since most of my time on web frameworks is spent in the Java world – and even then, I don’t do much interactive web apps. Mostly web APIs. APIs are so much easier to test.

Interactive web apps for humans are harder to test. This is where Capybara comes in, along with its collaborators Cucumber for the test description, and the integrations with Rack and Selenium WebDriver for the web testing backend.

While I know Selenium since it can be used for different target web platforms, I know less about Rack and Rails infrastructure. Fortunately, the book explains enough about these pieces for a relative newbie, and walks through worked examples that are more complete than the tools’ web documentation.

While Application Testing with Capybara is short, it goes straight to the point into what someone would need to know about the topic. Those with more time could rely on the free documentation.

Packt Publishing sent me a review copy of Instant Wijmo Widgets How-to.

I guess I was chosen since I worked with widgets before – but those were W3C-type Web Widget apps, in a different project. These Wijmo widgets appear to be widgets in the traditional sense – decorations for a website. They also seem to be based on jQuery.

To be updated with the actual review.

Update: I decided not to proceed with this review. The book has serious problems.

The publisher, Packt, gave me a review copy of this book.

Continuous Delivery and DevOps: A Quickstart Guide by Paul Swartout.

http://www.packtpub.com/devops-quickstart-guide/book

I am personally interested in DevOps. I have done both development and operations – and like being of value in both. I am immersed in software development – enterprise software at the job, and startup-type stuff as a hobby. Delivering is important. I am a fan of startups, where building things quickly and well is important. I am also aware of the problems raised by the contrasting priorities of Development and Operations.

The book opens with a story of a fictional “web-based software business” – or “cloud or SaaS,” since those types of setups are driving the culture and need for Continuous Delivery and DevOps. It walks through its evolution from a small shop to a mature organization through an acquisition. This allows the narrative to illustrate the conditions and problems that are specific to both types of company setups. The software delivery flow goes through three levels: a very simple one (Have a great idea -> Develop it -> Deploy it), an overly complicated one, and finally a reasonable flow given the evolution of the company. I feel the first chapter sets the tone that the book is trying to talk to everyone but might miss out on reaching to some audiences who won’t be able to relate. For example, while I believe the book should be readable by a non-technical project manager, it drops some jargon in an example.

The story moves on to identifying the “Elephant in the Room” and finding the causes of problems in software delivery by engaging the people involved. The investigation techniques are Value Stream Mapping, Retrospectives, and the Timeline Game – actual games. After the information is gathered, plans are made to setting and communicating the goals and vision, to lead the plan of attack on implementing Continuous Delivery and DevOps. This is done by standardizing the vocabulary and language (watch the jargon!), operating as a business change project, working with a dedicated team, acting as an evangelist, making decisions with courage, understanding the cost of change, and seeking advice from those with similar experience. I find that these actions, while all good,  belong to different roles and am not sure as to who the book is talking to.

After the setup, implementation is next. This is using source control (which is already standardized in many places, with the help of services like GitHub that promote it); keeping changes small, frequent and simple; having clean consumer/provider separation (software architecture advice, actually); open and honest peer working practices (tied up with corporate culture); failing fast through test-driven development; automated building and testing and continuous integration. The story then goes back out to architecture: how component-based architecture with loosely-coupled pieces works better, separated by layers of abstraction – clearly aimed at the software architect role. This is followed by advice on environments, tooling,  provisioning systems and monitoring – a common responsibility of Development and Operations.

The book then turns around to talk about culture and behaviors – a chapter that reads like an Agile Methodology advocacy mixed with Software Engineering Management and general management. A key point here is getting rid of a culture of blame, and bring it up to a culture of trust. While this is important for any organization, the speed DevOps brings highlights this need.

The last chapters talks about the hurdles that need to be overcome and ways to overcome them, and then measuring success and staying successful. This part reads like a software engineering management advice book, and not specific to Continuous Delivery and DevOps.

My main criticism of the book is that it rapidly switches between technical and non-technical topics. It doesn’t seem too sure of which audience it audience it wants to address. I believe that it would have been more effective if the technical and non-technical parts were separated into different sections such that the reader can easily choose the relevant topic. A lesser criticism is that the book often drifts into advocacy or evangelizing the need for Continuous Development and DevOps, when it may already be preaching to the choir who already believes in the concept.

The benefit of the book is that it gives some good starting points that the aspiring practitioner can take action on.

In the past couple of years, I have been reading a number of books. Almost all have been ebooks. the print medium may become obsolete, but the mental model of the book will not.

It’s about time I review them, reflect on them, and relate to my experiences.

This answer is so good it should be broken out of the Quora signup-wall.

Answer by Michael O. Church:

1. The processes that result in quality software are almost impossible to control or manage. There's an expectancy-variance tradeoff in creative work, including software. If you want greatness, you hire excellent people and give them full autonomy, with the understanding that you won't always get the features you want on the schedule you expect, but that everything that is actually important will be done by a very smart, competent person in time. (Think of it as akin to Eventual consistency (wikipedia.org). It's not always what you want, but if you can afford to wait, you get good results.) With this approach, you get excellent average-case performance (expectancy) but a lot of variability, especially in the short term. On the other hand, if you want reliable mediocrity, you hire a lower quality of engineer in larger numbers, and micromanage. Your expectancy drops, but you're minimizing variance. The first produces an R&D environment that produces great work in the long term, but has variable output and can be politically messy (people envy the engineers for having better jobs in their R&D environments, and eventually it becomes An Issue). The second is much more tenable in the typical corporate environment. You can define "deliverables" and know who to fire if the schedule slips.
2. Enterprise projects tend naturally toward Bigness, and Big Software is rarely of high quality. (See: Java Shop Politics (wordpress.com)). Competent software engineers like modularity, small programs, and simple interfaces. This is the essence of the Unix philosophy: large problems should be solved with systems and given the respect that systems require, and not with massive single programs with poorly specified and undocumented communication protocols. Good engineers make the software environment better by writing small programs– executables, utilities, libraries– and only create Big Things when absolutely necessary. However, this type of effort is hard to track, because one programmer might work on 20 different programs in a given year. Management has no idea what its best people are doing. Managerial dinosaurs feel more secure when the programmer:project relationship is reversed– many-to-one instead of one-to-many– because they can control the allocation of efforts when they get together and fight over "headcount". Unfortunately, Big Programs tend to collapse under the weight of their complexity, especially when they change hands.
3. Birdshit syndrome. This is related to #2. Imagine a square-mile canvas, laid down in a park, that you intend to paint on. Before you're done, it's overwhelmingly likely that it will be ruined by birdshit, on account of the massive size of the canvas. Enterprise software projects need to constantly market themselves within the organization, and the increasing visibility (surface area) also drives up the risk of attracting nonsensical requirements (birdshit) from on high. If the project seems to be at risk of becoming actually important, everyone who has the power to slow the project down expects a hand-out. These requirements are akin to the never-ending and often escalating outflow of bribes and extortions that are required to do business in a politically-undeveloped country. If there are 18 people who have the power to stop the software project, then there are 18 directions from which nonsensical feature expectations and requirements can come, usually before the project is established or mature enough for its managers to say "No". After a few years, the conceptual integrity of the project's original design is gone.
4. Lack of career incentives. Big Software projects are good for the manager's career, because he can put on his CV that he managed a large team. Having 20 reports is more impressive than having 4, regardless of whether the team of 4 engineers accomplished more. These projects are bad for engineers' careers, because per-person productivity is so low, and because you don't really learn much if you're not in a leadership or architectural role on such an effort. On Big Software projects, 10 to 20 lines of code per day is about average, 80 percent of the work is maintenance even before the thing launches, and it's not uncommon to go two months without checking in a line of code. Worse yet, the engineers gain none of the visibility that they might have if they, for example, maintained or started an open source project. This is a good environment for maxed-out, bored engineers who want to milk the clock for 5 years (long enough to make a manager-level external promotion inevitable) and not get fired, but the better and more ambitious engineers tend to realize when a project won't do anything for their careers, and they start planning their exits.
5. "Uncanny Valley." Good software engineers either want to be directly in the line of business in high-impact roles (startup founder, trading algorithms, data scientist, CTO) or they want to be far away from it in a blissful R&D land with full autonomy over how they spend their time. The first provides the way to higher social status and credibility, better compensation, and most importantly, the chance to have an impact. Work in the direct line-of-business is often unpleasant– even (if not especially) for high-status players– but the upshot is knowing that your work actually matters. The second category of environment (R&D) gives the engineer the right to direct his efforts and optimize for learning, career growth, and long-term impact. Most IT organizations live in an uncanny valley between these two states, where the engineers' lives and quality-of-work is affected by the line of business, but in which they're second-string players, treated as cost centers rather than profit generators. Good engineers, if they're going to accept the compromises that come with being in the line of business, also want the "key player" status and credibility that can come with it. Most IT organizations have the compromise but not the status.
6. Talent bleed. This is related to #4 and #5. The best engineers don't want to work on projects that have lost their conceptual integrity, because projects that get to that point are bad for an engineer's career. About 50% of those over-ambitious enterprise projects fail completely, and the other 40% are held up by a "too big to fail" sunk-cost mentality, but become disparaged legacy systems that not only harm the reputations of those who worked on them, but also erode the reputation of IT in general, creating the impression that "our engineers can never do anything right". (Weirdly, people who manage bad projects always seem to be OK. They "took one for the team" in attempting them, even though they delegated all the crap work.) When good engineers end up on these projects, they either negotiate for more autonomy (which often cannot be afforded) or more compensation (which is hard to get when the manager already has to fight hard for headcount). The only way a typical enterprise effort can keep its most talented people is to give them "special projects" (see #5) outside of the line of business, but very few IT managers have the "headcount" to afford that. Occasionally, however, one of these "special projects" will turn out to be really cool and take on a life of its own. Then, it needs additional support in order to meet the standards imposed by the business. Then it has to fight for headcount… the Cycle begins anew.
7. A vicious cycle of low status leading to crappy results. Because most companies can't keep talent in their IT organizations, the result is that these highly ambitious Big Software projects tend overwhelmingly toward mediocrity. They build all-or-nothing systems that just barely work, but people don't have the right not to use them (as would occur on the market) because that would be politically messy. In the long term, this elephant graveyard of crappy Big Software brings down the status of IT, and reduces its autonomy within the organization even further.

View Answer on Quora

Yesterday I got inspired by the 18th Anniversary of Philippine Internet and Netscape running on an old computer. I decided to try Mosaic on my Linux box. Mosaic, the predecessor of Netscape and so many other browsers.

Here is how I built it on Ubuntu Precise Pangolin (12.10) beta 64-bit:

• Downloaded the modernized Mosaic source code. The PNG patch is no longer available.
• Installed dependencies: sudo apt-get install libmotif-dev libxmu-dev libjpeg-dev libgd2-xpm-dev
• Manually downloaded and compiled lpng1058.tar.bz2from http://libpng.sourceforge.net . Good thing the old PNG libraries compatible with old source are still being maintained.
• Renamed the definitions ofgetline in ./libnut/url-utils.h ./libnut/url-utils.c ./libwww2/HTInit.c . It is already present in the standard C library.
• Compiled with make linux.
• The compile will have warnings about assigning pointers (64-bit) to int. While this is dangerous, it doesn’t stop this Mosaic from running – but it does segfault for some pages. It might be better on a 32-bit machine.
• The link will fail because the current libpng does not have the deprecated function png_read_init. Instead, copy paste the gcc -o Mosaic line and replace the -lpng with the compiled libpng.a. Also include the system’s libm.a (in my case, /usr/lib/x86_64-linux-gnu/libm.a) since libpng.a is statically compiled. Otherwise, the link will fail with undefined reference to `pow’.

Then, run ./Mosaic. It will have errors like Warning: Cannot convert string “-adobe-courier-medium-r-normal-*-14-*-*-*-*-*-iso8859-1″ to type FontStruct which don’t seem to prevent it from running.

On some sites, I get a core dump. I think it’s because the code was linked against the current libpng.h, but I don’t have time to investigate it. Take a look!

libpng warning: Application was compiled with png.h from libpng-1.0.6 or earlier
libpng warning: Application  is  running with png.c from libpng-1.0.58
libpng error: The png struct allocated by the application for reading is too small.

Trying WordPress for Android.

Twitter and Google+ killed light posting for me. Quora competes in more serious writing.

I need to find some good topics for writing.

I haven’t been updating here, so I should say: I’ve been doing most of my writing on Miguel Paraz’s Quora page.

Now testing Windows Live Writer. I hope, by keeping it open, I get tempted to write.

Trying to write again, after a year. This BlackBerry app for WordPress should help – I usually feel like writing when I’m far away from the computer.

I have been trying to reboot my writing, but nothing comes. Hope 2010 would be a more cultured year.