Why offshoring works

Contrary to the popular believe in developed world, the outsourcing actually works quite well.
Let's face it - if all companies were worse of after outsourcing, then the practice would have died long time ago. But it didn't. In fact, now even more companies are outsourcing their work to offshore companies (most to India and other parts of Asia). Nowadays, not just IT but every other sector like law, education, engineering etc. being outsource regularly.
Since I am on IT sector, I can only vouch what is happening in this segment.
The traditional IT jobs in UK are very much based on imposed responsibilities. It means, employees are designated to work particular task – like developer will only develop the code, tester will only test, support personnel will just execute (without much thinking) and Business Analysts will only look at Excel charts. Moreover, managers won't have a clue what their development team is working and System Architects will have no idea of what it feels like working in a particular technology.
Most UK IT firms refuse to accept the fact that IT sector works based on role based responsibilities rather than designated based ones!
In Indian IT companies, same person invariably works like developer, designer or even in project manager roles. Not only it helps to understand issues faced by other team members, it also avoids boredom.
Support jobs are always boring. Testing is like necessary evil – not many people want to do it yet it is absolute critical for any application to undergo rigorous testing. I hate the typical project manager's question – how much percentage of the work is actually completed. Gosh, when will they understand that it is not a linear relationship? For first few months, there may be only 10% of work done. But in last month alone 90% work can be completed. How is that possible? Quite easy. When a project begins, lots of questions are left unanswered. The application is developed with voids in between. Now that application is as such worthless as it can't be put on practice for its desired role. But, when the voids are fixed, then application becomes fit for purpose. Till it happens, it is very difficult to give a percentage completion report.
I remember, one of my former project manager stopped asking me this question when I replied – it is just 43.3976% done and progressing on a logarithmic scale.
Ok, coming back to our off-shoring topic. So far we covered boredom and role rotation philosophy. But that is not all.
In India, most development work is done by fresh graduates (out of universities). They are quite happy to work at lesser salary (even on local standard compared to experienced people there) but with not unacceptable quality – because most of them are quite eager to show their expertise in programming logic.
However, on western economies, since people tend to do same work for decades, they usually attract higher salaries (local standard) with out much quality improvement. One reason here is that you can't discriminated based on age – which is quite the opposite in outsourced countries.
Since off shoring companies employ fresh candidates without real life experience, there is always a chance that they will produce junk output. However, this is taken care by strict adherence to standards. It is shame that many UK companies just don't follow any standards at all – almost everything is ad hoc!
The managers probably here don't recognize that just by implementing rigorous standards alone, a lot of time and money can be saved.
Standard implementation is complemented via documentation. Since people working in project are separated by several hours of difference time zone and no face to face interaction is possible – the onsite system architects have to be extra cautious to avoid any ambiguity in design documents.
Yes, sometimes it means spoon feeding everything to offshore team. But it works in longer run. The design document becomes so much fool proof that anyone can just blindly follow them and get things done.
Of course, there is cost advantage as well. But what I discussed here are just some other aspects of off shoring.
I shall continue this discussion later.

How Google makes money?

I know, you will jump and say from advertisement, isn't it.
Yes, they make most of their revenue from advertisement - nothing special about it. But the question is how do they do it. Is it all fair for rest of us i.e. netizens?
You are already aware that when you search anything in Google, a colored horizontal and vertical (on right) column are displayed which says sponsored links. Obviously Google does take money from those companies for displaying their ads in web result. But users are clever and they rarely click on those links. Just ask yourself, how often do you click on those links?
But people, although the number is very small - approximately 1 click on such ads for every 1000 page visits! However, Google pages are accessed by millions (if not billions) of visitors every day. So, for Google, even 0.1% such clicks do bring huge amount of revenue. But sometimes Google does behave unethically. You can be a member of Google's adsense and put ads on your own blog (like this) and Google promises to send you money when user clicks ads on your page. But the catch is, you won't get payment until your account accumulates $100. For a small website, it will take years to reach that stage. So, all these years, Google will get revenue from advertisers but won't pay you anything for displaying them in your page. No worries, after you reach $100, you'll get the money - isn't it? Well, here lies the ugly bit. Google can (and they do) terminate your adsense account at any time if they think clicks are not legitimate. Still it seems reasonable, as you are not supposed to click on your own ads (i.e. they can track from which IP ads were clicked etc.) But even then, Google can terminate your account at anytime without giving you a reason at all. Well, they do - they did it with me. Then I searched on the net whether other people have similar experience and yes, they have done it with millions of people! They claim they would return money to advertisers but how do you know they have done it in practice?
Not very nice, is it?
The story does not end here. Every time you search anything or use Gmail, or Google Toolbar etc., Google stores information of your activity. There is no time limit how long Google will store that information. They can store it forever. Then they sell these information to 3rd parties for market analysis (does the word Google Analytics ring bell?).
Google bots scan your emails (if bots can do it, any one in Google office can do as well), it displays some ads on your inbox title bar based on what you typed in your email. Google's Chrome collects all bits of your web browsing info (that's why I don't use it but I still use Gmail though).
Google never discloses what algorithm it uses to check invalid clicks adsense account, how to rank websites, how it uses user net surfing info etc. In a nutshell, Google is never responsible to anyone for its activity.
Are you now afraid of searching anything using Google?
I am sure you will now ask me if I am recommending not to use Google at all. Well, it's your choice really. All I suggest that you don't put all your eggs in one basket. Always use some alternative emails, alternative search engines etc. That's it.

How online feedback affecting business

How do you search for a product? The product may be your next car, enterprise database in your organization or finding best re-mortgage rate. Chances are (well, I'm pretty confident) that you search for best deals on internet.
You are most likely to scan three types of sites - viz
1. Each vendors' own website

2. A comparison site (often known as screen-scrappers like money supermarket, froogle etc.)
3. The feedback written by general public (either online or anecdotes from your friends, colleagues etc.)
In pre-internet era (say before 2000 when internet was still few and far between), online feedback sites were rare and so was comparion sites. So, consumers could only gather information from respective company's websites and whatever they claimed in traditional TV, radio or newspaper ads.
The obvious problem was to validate their claims. If they said "according to XYZ survey conducted..." you really not have much scope of validating this claim. You don't really have enough time/resource to find out the research report from library journals.
But now, thanks to online forums, companies don't have space to hide. The best example, is to select a car. Almost all manufacturers have user forums on internet. Before you actually test drive a car, you have access to all the things that went wrong with particular model. So what? Does it prevent anyone buying a lemon or selecting wrong mortgage?

No - of course it does not completely eliminate bad selection, but no doubt it makes consumer a lot more wiser and apprehensive of what to look for.

Companies often claim that for every bad review, there are hundreds of satisfied customers. Yes, it is quite true on some extent. Often consumers won't praise a product often as they criticise it when things to wrong. So, no review of a product can have two meaning - either everyone using it was very satisfied or it sold so few that even those with bad experience, didn't bother writing their feedback online.
What is interesting is that, even today, many big companies are not really taking online feedback seriously. A well known case was "Dell Hell" (search for it on internet). Basically few years back Dell started selling crap laptops and people flooded the web with poor consumer experience. That seriously did a dent on Dell's revenue. Finally companied take steps to raise their quality and pacify the online rants of users.
Now the question which begs an answer is that why companies are still ignoring online citicism from users? It depends. If the company's primary user base is old people - who are not net savvy, they know they won't hit hard as mainly younger generations are glued to net. Many people, often confuse anectodal evidence (ie. my friends car worked fine for 10 years) with statistical evidence (eg. 500 models of same car had a serious breakdown in first year).
There are some agencies, who actually work professionally to improve online reputation of many companies! So, if company A sees that complaints about their product is flooding the net, then they ask for these specialists to counteract the issue. These firms then try to dilute negative feedback with loads of artificial positive reviews. They even employ tactics so that search engines brings up good feedbacks first.
It's a cat and mouse game.

Cost of ETL tools

Did you recall the cost of databases chapter? Well, getting price of ETL tools even harder work.

There was not a single ETL vendor (other than open source ones) who feature their price on their website.

If you ring them up for their price, they will ask you so many questions you will be confused more than ever (it is as irritating as answering questions while you apply for your car insurance).

Some ETL vendors are very snob! They may refuse to sell you their product unless they think your organization is qualified enough to use their product! This is definitely not a problem for multinational enterprise companies but it is indeed a problem for small/medium enterprises that don’t have that much budget or exposure. It will be interesting to observe if recent economic downturn would have changed their outlook (nowadays they sometimes do give freebies like an extra product thrown with no cost).

All I can tell you that major ETL software (Ab Initio, Informatica, Data Stage) licenses do cost in the region of ~ $500,000 for a typical large organization.

The table below gives you an approximate idea of cost.

ETL	Price	Demo/ Try before buy
Ab Initio	$ 250,000 for Co-operating system per CPU $ 10,000 per GDE $ 500,000 for EME now their price is based on Specint (faster processor = you pay more)	No
Informatica	? + mandatory database cost for repository	Yes, you can download working version from Oracle’s site (yes Oracle’s)
Data Stage	$ 55,000 - $ 110,000 per CPU	?
Oracle Warehouse Builder	?	Yes, you can download working version from Oracle’s site
Talend Open Studio	$ 10,000 per year per user	Yes
Expressor	Depends on channel which is a confusing concept introduced by them. See their website.	No

Photography - basic techniques revisited

In my school days I started to use traditional manual film camera. But after the invention of digital cameras, I chucked away my manual camera and plunged into point & shoot mode. Over a decade, I never thought of going back to manual camera as I don't have patient to adjust every time. However, when my compact camera finally gave up its soul, I had look for another one. I thought of buying a Digital SLR. But money was a constraint! So, finally settled down for a semi automatic digital camera (also known as Bridge camera).

As I was revising my long forgotten camera knowledge, thought of writing it down here in concise form so that I can refer it in future.

Film speed = measured in numbers like 50, 64, 80, 100, 200, 400, 1600, 3200, 6400. Lower the number, less grain/noise captured in photo (which is good). You should always try to shoot images in lowest ISO speed possible for best quality of image. However, higher ISO films are most sensitive to low light and so you need to use higher film speeds (200 and above) in indoor or low light conditions. Outdoor photos in sunny days should be shot at 100 ISO or lower.

Aperture = the measurement of opening of the lens. Measured in F#. Higher F# means smaller opening and vice versa.
For example, F1.8 will allow more light (= more exposure) to enter on camera compared to that of F16.

Shutter speed = duration of lens opening. Longer duration means more exposure ( = more light) and vice versa. Measured in fraction of second. 1/60 shutter speed means shutter will open for 1/60th of a second.

Aperture and shutter speed relationship as followed in most compact digital cameras. [Not entirely sure about this tabulation - I need to double check]

F1.8 F2.8 F4 F5.6 F8 F11 F16
1/250 1/125 1/60 1/30 1/15 1/8 1/4

Shutter speed range - 4/1 to 1/2000 s

Large aperture = smaller F# = more light = shallow depth of field (subject is nearer) = only subject in focus = portrait

Smaller aperture = larger F# = less light = distant depth of field (subject is far away) = everything in focus = landscape

Aperture priority mode = you set aperture value and camera decides appropriate shutter speed
Shutter speed priority mode = you set shutter speed value and camera decides appropriate aperture
Manual mode = you select both aperture and shutter speed

Lens focal length

Compared to traditional 35 mm cameras, lens of 28 mm focal length or below is considered in wide angle territory. Lens below 20 mm is very good wide angle. From 11 to 18, it is considered ultra wide angle. At extreme, 6 mm lens is known as fish eye. Very wide angle lens do show barrel distortion.

Standard fit lens of most SLR cameras vary from 10 to 60 mm. So, most of them have built in wide angle lens. They use separate lenses for telephoto/zoom.

CCD sensor - This is equivalent of film in digital cameras. Larger the CCD sensor size, more details can be captured. Most digital compact cameras has small CCD sensor (4x6 mm). The DSLR cameras have much bigger CCD sensor size (~ 16x24 mm) for which they can capture rich vividly colored images. CCD means Charged Couple Device. Traditional 35 mm films have size of 24x36 mm. Larger CCD is more expensive. Sensor size difference is the fundamental reason why DSLR picture quality is far better than those taken by digital compact cameras.

Shooting techniques

Panorama

All images, using which you will stitch panorama, must have uniform exposure. Otherwise they will look awkward (with different brightness in final panorama) or you painstaking photo-editing to adjust exposure in your computer. Any decent panorama sticher software will be able to stitch images to generate stunning looking panorama.

To ensure uniform exposure, you must lock exposures between images (if your camera allows this). Otherwise, if you camera has panorama assist mode, it will do this for you by itself. If you camera has neither, they you can take panoramas only in bright sunlight (without any shades) where your camera is unlikely to vary exposure between shots.

Landscape

For landcape shooting in bright sunlight, use higher F# (over F5 possibly F11 or F16) and lower shutter speed. Subject is considered at infinite length for this shot. Keep flash off. Use lowest film speed possible. When you select Landscape mode in compact cameras, internally they adjust settings like this. If sky is overcast, use a slower shutter speed and/or larger aperture (= lower F#).

Portrait

Subject should be in focus. Here you have shallow depth of field, so use lower F# like F2.7. Adjust film and shutter speed depending on where you shoot in outside or indoor. For night indoor, you might need to use flash.

Moving objects

Since the object is moving (eg. moving car or athlete), you need to use a high value of shutter speed (typically 1/250 or faster depending on speed of object). Many compact cameras have sport mode for this shots. Some cameras offer burst mode - where you can press the shutter button for sometime and camera takes photos continuously.

Environmental factors

The best photos are shot in bright sunlight! In fact, even a very cheap camera takes brilliant shots in sun. The capability of camera shows up during overcast days, indoor shots and super zoom (macro) levels.

Mega Pixel (and myth)

All cameras now advertised with megapixel values. Does a higher megapixel means better image? A megapixel is calculated as = (# of horizontal pixels) x (# of vertical pixels) / (1024 x 1024).

So, an image size of 2048 x 1536 / (1024x1024) is 3 megapixels.

640x480 = 0.3 MP = VGA quality
3624 x 2448 = 7.6 MP etc.

All the pixels are constrained by CCD sensor size. For most digital compact cameras, the sensor size is quite small. For same sensor size, an 8-MP camera will have more number of smaller pixels compared to a 3-MP camera - but distributed over same area! If you peek your nose over a television screen, you will see lots of tiny dots. These are pixels. More megapixel means smaller dots. If the color changes between neighboring dots, smaller the dots are, smoother the changes will appear. That's the story behind higher megapixels. However, a true TV viewing experience does not always depend on resolution (MP for camera) but also on how big the screen is (say 42 inch screen against 26 inch LCD). The screen size is equivalent to CCD sensor size!

Now see the difference, when people talk about TVs, they measure screen size but when talking of cameras, they don't talk sensor size but number of pixels! Otherwise how would camera manufacturers make you believe more MP means better?

Thus the correct comparison for quality of image between two cameras is this = square root of (higher MP / lower MP).

For a 3-MP and a 6-MP camera, quality difference comes to = sqrt(6/3) = 1.41. So, one is only 41% better than other not the 200% as media make you believe! Buyers beware!

Sample specification of the Fujifilm S8000fd camera, which I now use

ISO film speed - 64/80/100 to 6400 in double increment
Aperture range - F2.8 to F8
Shutter speed range - 4/1 to 1/2000 s
Lens range (35 mm equivalent) - 27 to 486 mm

Do you really need an ETL tool?

Most ETL tools are quite expensive (especially those which actually work in very large DWH systems).

Many small and medium size organizations perform ETL operations via regular programming languages viz. C++ or PL/SQL.

White it may not be the most efficient and productive way of ETL processing, we must remember the cost benefit analysis.

In a project where source and target databases are both the same (say Oracle), it makes perfect sense to carry out moderate ETL processing via PL/SQL alone as long as you get acceptable performance out of it.

The requirement of a full scale ETL tool most felt in a heterogeneous environment.

The main disadvantage of using traditional programming languages for ETL work is the performance. Programming languages tend process data in loops. For small data volume it is fine. However, looping technique is not very performing for large data sets. Yes, it is possible to tailor made your procedural language code to take advantage of parallelism (tormenting in C++ but bearable in PL/SQL) but in reality it is as involving as writing an ETL tool yourself and thus not really rewarding experience (unless you can market your tool).

In ETL tools, you don’t have to do anything (except keeping your data organized across partitions etc.) to gain performance advantage. They are designed to take parallelism etc. in their execution plan.

But in procedural language, you are of your own. One way to improve performance in PL/SQL (or similar) is to write cursors less often and use set operations more often. Most databases are designed to use best parallelism available when you use set operations.

Don’t forget cost of recruiting personnel in your chosen ETL tool. Depending on market condition, experts in one ETL tool might demand more compensation than other one. You also need to budget for organizing training for your existing staffs.

Most multinational companies can easily afford the best tool sets. However, for medium sized firms, budget is always a constraint. So, they are more prone to in-house development rather than buying tool set of the shelf.

How internet & Ebooks changing the way we read books

Ok, let's admit - internet and Ebook have revolutionized the way we read books. Of course, lot of people (including myself) still read traditional books but we are reading more and more books/magazines/newspapers on the screen!

I still read non-technical books in conventional book format, but prefer to read the technical books (mostly computer related because of my profession) on internet or Ebook format. The obvious advantage is that I can copy and paste any commands on my application instantly and check the benefit.

Sites like www.scribd.com or www.lulu.com (and many similar sites) have indeed changed the landscape of book writing and publication.

I believe the profit from publishing book or royalty from the sales are gradually diminishing over the years. In the past, the only way to read a book was to either buy it or borrow from library. But now, a lot of books are either available free of charge on internet or they cost a minuscule amount.

I am not arguing with the fact that sometimes copyrighted versions of some books are available on the internet which is illegal, but the bottom line is you can still download a very good book at no cost to you.

Most Ebook readers are still expensive. However, you can read ebooks on PDF format in most PDAs and small net book computers. In near future, the Ebook readers' price will only go down. More and more authors will publish their books in ebook form. Some ebooks are in proprietary format. However, as the ebooks will gain more popularity, there will be some hackers who will always make things breakable (again I am not going in the debate of how un-ethical is that).


Gone the days when people used to consult giant technical manuals when something did not work in software. Now people search on internet and unless the problem is too difficult or uncommon, usually within few minutes a solution can be found and problem is solved.


Even if problem is not found on the web search, you can often raise it on discussion forums and people all over the world in your profession will browse and try to give you an answer. This is like having access to lots of technical experts at your finger tip. I admit that sometimes you do get crap suggestions for your problem, but most of the time, the tricks do work and people really help each other.


Perhaps the funniest part is in newspapers! Gone the days when I avidly read first page on your favourite newspaper. Now you already know what will be on the first page - as you have seen the breaking news on previous evenings new sites! This is one reason why newspaper circulation in developed world is dwindling down every year. Another reason is obviously, the high cost of buying a newspaper. £1.20 a newspaper a day is not cheap!

From the consumer's perspective, though, there is a huge difference between cheap and free. Give a product away and it can go viral. Charge a single cent for it and you're in an entirely different business, one of clawing and scratching for every customer. The psychology of "free" is powerful indeed, as any marketer will tell you. From C. Andersen's blog - the author of Long Tail theory. However, I do wonder whether the author is willing to give away his next title "Free" for free.


We shall see how rise of "freeconomics" and "freemium" products continue to affect our lives.