Theory of Everything

Fundamental particles and forces

The most common force of nature is gravitational force! Newton's famous 3 equations of motions are based on gravitational force.

This law basically states that two objects in universe attract each other at a force defined by this universal law of gravitation equation:

F = G*m1*m2/ d^2

Where, G = 6.673 x 10^-11 Nm² kg^-2

m₁ = mass of first object (kg)

m₂ = mass of second object (kg)

d = distance between two objects (m)

F = gravitational attraction force (N)

This equation is applicable to objects which are small (few mm) to infinite distance apart.

It pulls matter together, causes you to have a weight, apples to fall from trees, keeps the Moon in its orbit around the Earth, the planets confined in their orbits around the Sun, and binds together galaxies in clusters.

Classical mechanics (or Newtonian physics) laws are applicable to cases where size of particles are larger than atomic level and speed of travel is much lower than that of light.

If you are looking at physics of particles at atomic level, then you need quantum theory. If you are approaching near speed of light, you need Einstein’s theory of relativity.

Example, two cars are coming towards each other at a speed of 30 km/h each their relative speed is 30 + 30 = 60 km/h.

However, if two spaceships are travelling at 270,000 km/s each their relative speed will be found using following formula

u = (v + w)/(1 + v*w/c^2)

Where c is velocity of light, v and w are speeds of two spaceships measured by a 3^rd observer. Note that for low values of v & w, the above equation becomes u = v + w.

Atom = nucleus ( = protons + neutrons) + electrons

Electrons have -ve charge while protons have +ve charge.

Since particles with same charge must repulse from each other, there must be a force which holds same +ve charged protons together at nucleus. This force is known as strong nuclear force. This force acts only in a atomic range (10^-15 m).

Because the strong force binds nuclear particles so tightly together, huge amounts of energy are released when lightweight nuclei are fused together (fusion reaction) or heavy nuclei are broken apart (fission reaction).. The strong nuclear force interaction is the underlying source of the vast quantities of energy that are liberated by the nuclear reactions that power the stars.

The force for which opposite charged particles (eg. protons & electrons) repels from each other, is known as electro magnetic force.

Which is defined by Coulomb’s equation

F = k1 * k2 / d^2

Electrons are attached to nucleus by this force – thus it holds atoms and molecules together! Like gravitational force, electromagnetic force is applicable to an infinite range.

The electromagnetic force binds [negatively charged] electrons into their orbital shells, around the positively charged nucleus of an atom. This force holds the atoms together.

The electromagnetic force controls the behavior of charged particles and plasmas (a plasma is a mixture of equal numbers of positive ions and negative electrons) as, for example, in solar prominences, coronal loops, flares, and other kinds of solar activity.

The electromagnetic force also governs the emission and absorption of light and other forms of electromagnetic radiation. Light is emitted when a charged particle is accelerated (for example, when an electron passes close to an ion, or interacts with a magnetic field) or when an electron drops down from a higher to a lower energy level of an atom (from an outer to an inner 'orbit' around the atomic nucleus).

The weak nuclear force causes the radioactive decay of certain particular atomic nuclei. In particular, this force governs the process called beta decay whereby a neutron breaks up spontaneously into a proton, and electron and an antineutrino. If a neutron within an atomic nucleus decays in this way, the nucleus emits an electron (otherwise known as a beta particle) and the neutron transforms into a proton. This increases (by one) the number of protons in that nucleus, thereby changing its atomic number and transforming it into the nucleus of a different chemical element. This force is applicable to atomic (10^-17 m) range only.

The weak force is responsible for synthesizing different chemical elements in stars and in supernova explosions, through processes involving the capture and decay of neutrons.

When confined within a stable (non-radioactive) atomic nucleus, a neutron is a stable, long-lived particle. Once removed from an atomic nucleus, a free neutron will undergo beta decay, typically in about twenty minutes. The reverse process of beta decay occurs in the collapsing cores of supernovae, where protons and neutrons are fused together to create the vast numbers of neutrons that populate the end product of the collapse - a neutron star.

It has been proved recently that electromagnetic force and weak nuclear force are basically same force – called electro weak force.

Before we move on, there are some other theories we need to learn. One among that is

quantum mechanics. The key idea here is that the smaller the scale at which you look at the world, the more random things become. Heisenberg's uncertainty principle is a famous example of this. The principle states that when you consider a moving particle, for example an electron orbiting the nucleus of an atom, you can never ever measure both its position and its momentum as accurately as you like. Looking at space at a minuscule scale may allow you to measure position with a lot of accuracy, but there won't be much you can say about momentum. This isn't because your measuring instruments are imprecise. There simply isn't a "true" value of momentum, but a whole range of values that the momentum can take, each with a certain probability. In short, there is randomness. This randomness appears when we look at particles at a small enough scale. The smaller one looks, the more random things become!

A major difference between quantum mechanics and classical mechanics is that law of physics depends on sizes of particles in very small scale dimensions. At very small scale (atomic level) the classical laws of physics no longer work. That is the place where quantum mechanics laws will apply.

The theory of everything

We now aim to define earlier fundamental forces by a single theory! That’s where string theory comes.

Theory of everything = to combine, quantum mechanics + relativity + classical mechanics

Classical physics assumes that a particle is the smallest existence. String theory asserts that the fundamental building blocks of nature are not like points, but like strings: they have extension, in other words they have length. And that length dictates the smallest scale at which we can see the world. The estimated size of these strings is 10-34 m.

The mathematics behind string theory is very complex! Our conventional physics so far assumed only 4 dimensions – namely length, width, height and time. But string theory, in its various forms, assume dimension of 10, 11 or even 26. However, it is stated that most dimensions actually exists only in atomic range distance.

Strings can vibrate. In fact they can vibrate in an infinite number of different ways. This suggests that the different particles and forces are just the fundamental strings vibrating in a multitude of different ways.

In a nutshell, the string theory gives us an exciting vision of nature as miniscule bits of vibrating strings in a space with hidden curled-up dimensions.

Why offshoring works

Contrary to the popular believe in developed world, the outsourcing actually works quite well.
Let's face it - if all companies were worse of after outsourcing, then the practice would have died long time ago. But it didn't. In fact, now even more companies are outsourcing their work to offshore companies (most to India and other parts of Asia). Nowadays, not just IT but every other sector like law, education, engineering etc. being outsource regularly.
Since I am on IT sector, I can only vouch what is happening in this segment.
The traditional IT jobs in UK are very much based on imposed responsibilities. It means, employees are designated to work particular task – like developer will only develop the code, tester will only test, support personnel will just execute (without much thinking) and Business Analysts will only look at Excel charts. Moreover, managers won't have a clue what their development team is working and System Architects will have no idea of what it feels like working in a particular technology.
Most UK IT firms refuse to accept the fact that IT sector works based on role based responsibilities rather than designated based ones!
In Indian IT companies, same person invariably works like developer, designer or even in project manager roles. Not only it helps to understand issues faced by other team members, it also avoids boredom.
Support jobs are always boring. Testing is like necessary evil – not many people want to do it yet it is absolute critical for any application to undergo rigorous testing. I hate the typical project manager's question – how much percentage of the work is actually completed. Gosh, when will they understand that it is not a linear relationship? For first few months, there may be only 10% of work done. But in last month alone 90% work can be completed. How is that possible? Quite easy. When a project begins, lots of questions are left unanswered. The application is developed with voids in between. Now that application is as such worthless as it can't be put on practice for its desired role. But, when the voids are fixed, then application becomes fit for purpose. Till it happens, it is very difficult to give a percentage completion report.
I remember, one of my former project manager stopped asking me this question when I replied – it is just 43.3976% done and progressing on a logarithmic scale.
Ok, coming back to our off-shoring topic. So far we covered boredom and role rotation philosophy. But that is not all.
In India, most development work is done by fresh graduates (out of universities). They are quite happy to work at lesser salary (even on local standard compared to experienced people there) but with not unacceptable quality – because most of them are quite eager to show their expertise in programming logic.
However, on western economies, since people tend to do same work for decades, they usually attract higher salaries (local standard) with out much quality improvement. One reason here is that you can't discriminated based on age – which is quite the opposite in outsourced countries.
Since off shoring companies employ fresh candidates without real life experience, there is always a chance that they will produce junk output. However, this is taken care by strict adherence to standards. It is shame that many UK companies just don't follow any standards at all – almost everything is ad hoc!
The managers probably here don't recognize that just by implementing rigorous standards alone, a lot of time and money can be saved.
Standard implementation is complemented via documentation. Since people working in project are separated by several hours of difference time zone and no face to face interaction is possible – the onsite system architects have to be extra cautious to avoid any ambiguity in design documents.
Yes, sometimes it means spoon feeding everything to offshore team. But it works in longer run. The design document becomes so much fool proof that anyone can just blindly follow them and get things done.
Of course, there is cost advantage as well. But what I discussed here are just some other aspects of off shoring.
I shall continue this discussion later.

How Google makes money?

I know, you will jump and say from advertisement, isn't it.
Yes, they make most of their revenue from advertisement - nothing special about it. But the question is how do they do it. Is it all fair for rest of us i.e. netizens?
You are already aware that when you search anything in Google, a colored horizontal and vertical (on right) column are displayed which says sponsored links. Obviously Google does take money from those companies for displaying their ads in web result. But users are clever and they rarely click on those links. Just ask yourself, how often do you click on those links?
But people, although the number is very small - approximately 1 click on such ads for every 1000 page visits! However, Google pages are accessed by millions (if not billions) of visitors every day. So, for Google, even 0.1% such clicks do bring huge amount of revenue. But sometimes Google does behave unethically. You can be a member of Google's adsense and put ads on your own blog (like this) and Google promises to send you money when user clicks ads on your page. But the catch is, you won't get payment until your account accumulates $100. For a small website, it will take years to reach that stage. So, all these years, Google will get revenue from advertisers but won't pay you anything for displaying them in your page. No worries, after you reach $100, you'll get the money - isn't it? Well, here lies the ugly bit. Google can (and they do) terminate your adsense account at any time if they think clicks are not legitimate. Still it seems reasonable, as you are not supposed to click on your own ads (i.e. they can track from which IP ads were clicked etc.) But even then, Google can terminate your account at anytime without giving you a reason at all. Well, they do - they did it with me. Then I searched on the net whether other people have similar experience and yes, they have done it with millions of people! They claim they would return money to advertisers but how do you know they have done it in practice?
Not very nice, is it?
The story does not end here. Every time you search anything or use Gmail, or Google Toolbar etc., Google stores information of your activity. There is no time limit how long Google will store that information. They can store it forever. Then they sell these information to 3rd parties for market analysis (does the word Google Analytics ring bell?).
Google bots scan your emails (if bots can do it, any one in Google office can do as well), it displays some ads on your inbox title bar based on what you typed in your email. Google's Chrome collects all bits of your web browsing info (that's why I don't use it but I still use Gmail though).
Google never discloses what algorithm it uses to check invalid clicks adsense account, how to rank websites, how it uses user net surfing info etc. In a nutshell, Google is never responsible to anyone for its activity.
Are you now afraid of searching anything using Google?
I am sure you will now ask me if I am recommending not to use Google at all. Well, it's your choice really. All I suggest that you don't put all your eggs in one basket. Always use some alternative emails, alternative search engines etc. That's it.

How online feedback affecting business

How do you search for a product? The product may be your next car, enterprise database in your organization or finding best re-mortgage rate. Chances are (well, I'm pretty confident) that you search for best deals on internet.
You are most likely to scan three types of sites - viz
1. Each vendors' own website

2. A comparison site (often known as screen-scrappers like money supermarket, froogle etc.)
3. The feedback written by general public (either online or anecdotes from your friends, colleagues etc.)
In pre-internet era (say before 2000 when internet was still few and far between), online feedback sites were rare and so was comparion sites. So, consumers could only gather information from respective company's websites and whatever they claimed in traditional TV, radio or newspaper ads.
The obvious problem was to validate their claims. If they said "according to XYZ survey conducted..." you really not have much scope of validating this claim. You don't really have enough time/resource to find out the research report from library journals.
But now, thanks to online forums, companies don't have space to hide. The best example, is to select a car. Almost all manufacturers have user forums on internet. Before you actually test drive a car, you have access to all the things that went wrong with particular model. So what? Does it prevent anyone buying a lemon or selecting wrong mortgage?

No - of course it does not completely eliminate bad selection, but no doubt it makes consumer a lot more wiser and apprehensive of what to look for.

Companies often claim that for every bad review, there are hundreds of satisfied customers. Yes, it is quite true on some extent. Often consumers won't praise a product often as they criticise it when things to wrong. So, no review of a product can have two meaning - either everyone using it was very satisfied or it sold so few that even those with bad experience, didn't bother writing their feedback online.
What is interesting is that, even today, many big companies are not really taking online feedback seriously. A well known case was "Dell Hell" (search for it on internet). Basically few years back Dell started selling crap laptops and people flooded the web with poor consumer experience. That seriously did a dent on Dell's revenue. Finally companied take steps to raise their quality and pacify the online rants of users.
Now the question which begs an answer is that why companies are still ignoring online citicism from users? It depends. If the company's primary user base is old people - who are not net savvy, they know they won't hit hard as mainly younger generations are glued to net. Many people, often confuse anectodal evidence (ie. my friends car worked fine for 10 years) with statistical evidence (eg. 500 models of same car had a serious breakdown in first year).
There are some agencies, who actually work professionally to improve online reputation of many companies! So, if company A sees that complaints about their product is flooding the net, then they ask for these specialists to counteract the issue. These firms then try to dilute negative feedback with loads of artificial positive reviews. They even employ tactics so that search engines brings up good feedbacks first.
It's a cat and mouse game.

Cost of ETL tools

Did you recall the cost of databases chapter? Well, getting price of ETL tools even harder work.

There was not a single ETL vendor (other than open source ones) who feature their price on their website.

If you ring them up for their price, they will ask you so many questions you will be confused more than ever (it is as irritating as answering questions while you apply for your car insurance).

Some ETL vendors are very snob! They may refuse to sell you their product unless they think your organization is qualified enough to use their product! This is definitely not a problem for multinational enterprise companies but it is indeed a problem for small/medium enterprises that don’t have that much budget or exposure. It will be interesting to observe if recent economic downturn would have changed their outlook (nowadays they sometimes do give freebies like an extra product thrown with no cost).

All I can tell you that major ETL software (Ab Initio, Informatica, Data Stage) licenses do cost in the region of ~ $500,000 for a typical large organization.

The table below gives you an approximate idea of cost.

ETL	Price	Demo/ Try before buy
Ab Initio	$ 250,000 for Co-operating system per CPU $ 10,000 per GDE $ 500,000 for EME now their price is based on Specint (faster processor = you pay more)	No
Informatica	? + mandatory database cost for repository	Yes, you can download working version from Oracle’s site (yes Oracle’s)
Data Stage	$ 55,000 - $ 110,000 per CPU	?
Oracle Warehouse Builder	?	Yes, you can download working version from Oracle’s site
Talend Open Studio	$ 10,000 per year per user	Yes
Expressor	Depends on channel which is a confusing concept introduced by them. See their website.	No

Photography - basic techniques revisited

In my school days I started to use traditional manual film camera. But after the invention of digital cameras, I chucked away my manual camera and plunged into point & shoot mode. Over a decade, I never thought of going back to manual camera as I don't have patient to adjust every time. However, when my compact camera finally gave up its soul, I had look for another one. I thought of buying a Digital SLR. But money was a constraint! So, finally settled down for a semi automatic digital camera (also known as Bridge camera).

As I was revising my long forgotten camera knowledge, thought of writing it down here in concise form so that I can refer it in future.

Film speed = measured in numbers like 50, 64, 80, 100, 200, 400, 1600, 3200, 6400. Lower the number, less grain/noise captured in photo (which is good). You should always try to shoot images in lowest ISO speed possible for best quality of image. However, higher ISO films are most sensitive to low light and so you need to use higher film speeds (200 and above) in indoor or low light conditions. Outdoor photos in sunny days should be shot at 100 ISO or lower.

Aperture = the measurement of opening of the lens. Measured in F#. Higher F# means smaller opening and vice versa.
For example, F1.8 will allow more light (= more exposure) to enter on camera compared to that of F16.

Shutter speed = duration of lens opening. Longer duration means more exposure ( = more light) and vice versa. Measured in fraction of second. 1/60 shutter speed means shutter will open for 1/60th of a second.

Aperture and shutter speed relationship as followed in most compact digital cameras. [Not entirely sure about this tabulation - I need to double check]

F1.8 F2.8 F4 F5.6 F8 F11 F16
1/250 1/125 1/60 1/30 1/15 1/8 1/4

Shutter speed range - 4/1 to 1/2000 s

Large aperture = smaller F# = more light = shallow depth of field (subject is nearer) = only subject in focus = portrait

Smaller aperture = larger F# = less light = distant depth of field (subject is far away) = everything in focus = landscape

Aperture priority mode = you set aperture value and camera decides appropriate shutter speed
Shutter speed priority mode = you set shutter speed value and camera decides appropriate aperture
Manual mode = you select both aperture and shutter speed

Lens focal length

Compared to traditional 35 mm cameras, lens of 28 mm focal length or below is considered in wide angle territory. Lens below 20 mm is very good wide angle. From 11 to 18, it is considered ultra wide angle. At extreme, 6 mm lens is known as fish eye. Very wide angle lens do show barrel distortion.

Standard fit lens of most SLR cameras vary from 10 to 60 mm. So, most of them have built in wide angle lens. They use separate lenses for telephoto/zoom.

CCD sensor - This is equivalent of film in digital cameras. Larger the CCD sensor size, more details can be captured. Most digital compact cameras has small CCD sensor (4x6 mm). The DSLR cameras have much bigger CCD sensor size (~ 16x24 mm) for which they can capture rich vividly colored images. CCD means Charged Couple Device. Traditional 35 mm films have size of 24x36 mm. Larger CCD is more expensive. Sensor size difference is the fundamental reason why DSLR picture quality is far better than those taken by digital compact cameras.

Shooting techniques

Panorama

All images, using which you will stitch panorama, must have uniform exposure. Otherwise they will look awkward (with different brightness in final panorama) or you painstaking photo-editing to adjust exposure in your computer. Any decent panorama sticher software will be able to stitch images to generate stunning looking panorama.

To ensure uniform exposure, you must lock exposures between images (if your camera allows this). Otherwise, if you camera has panorama assist mode, it will do this for you by itself. If you camera has neither, they you can take panoramas only in bright sunlight (without any shades) where your camera is unlikely to vary exposure between shots.

Landscape

For landcape shooting in bright sunlight, use higher F# (over F5 possibly F11 or F16) and lower shutter speed. Subject is considered at infinite length for this shot. Keep flash off. Use lowest film speed possible. When you select Landscape mode in compact cameras, internally they adjust settings like this. If sky is overcast, use a slower shutter speed and/or larger aperture (= lower F#).

Portrait

Subject should be in focus. Here you have shallow depth of field, so use lower F# like F2.7. Adjust film and shutter speed depending on where you shoot in outside or indoor. For night indoor, you might need to use flash.

Moving objects

Since the object is moving (eg. moving car or athlete), you need to use a high value of shutter speed (typically 1/250 or faster depending on speed of object). Many compact cameras have sport mode for this shots. Some cameras offer burst mode - where you can press the shutter button for sometime and camera takes photos continuously.

Environmental factors

The best photos are shot in bright sunlight! In fact, even a very cheap camera takes brilliant shots in sun. The capability of camera shows up during overcast days, indoor shots and super zoom (macro) levels.

Mega Pixel (and myth)

All cameras now advertised with megapixel values. Does a higher megapixel means better image? A megapixel is calculated as = (# of horizontal pixels) x (# of vertical pixels) / (1024 x 1024).

So, an image size of 2048 x 1536 / (1024x1024) is 3 megapixels.

640x480 = 0.3 MP = VGA quality
3624 x 2448 = 7.6 MP etc.

All the pixels are constrained by CCD sensor size. For most digital compact cameras, the sensor size is quite small. For same sensor size, an 8-MP camera will have more number of smaller pixels compared to a 3-MP camera - but distributed over same area! If you peek your nose over a television screen, you will see lots of tiny dots. These are pixels. More megapixel means smaller dots. If the color changes between neighboring dots, smaller the dots are, smoother the changes will appear. That's the story behind higher megapixels. However, a true TV viewing experience does not always depend on resolution (MP for camera) but also on how big the screen is (say 42 inch screen against 26 inch LCD). The screen size is equivalent to CCD sensor size!

Now see the difference, when people talk about TVs, they measure screen size but when talking of cameras, they don't talk sensor size but number of pixels! Otherwise how would camera manufacturers make you believe more MP means better?

Thus the correct comparison for quality of image between two cameras is this = square root of (higher MP / lower MP).

For a 3-MP and a 6-MP camera, quality difference comes to = sqrt(6/3) = 1.41. So, one is only 41% better than other not the 200% as media make you believe! Buyers beware!

Sample specification of the Fujifilm S8000fd camera, which I now use

ISO film speed - 64/80/100 to 6400 in double increment
Aperture range - F2.8 to F8
Shutter speed range - 4/1 to 1/2000 s
Lens range (35 mm equivalent) - 27 to 486 mm

Do you really need an ETL tool?

Most ETL tools are quite expensive (especially those which actually work in very large DWH systems).

Many small and medium size organizations perform ETL operations via regular programming languages viz. C++ or PL/SQL.

White it may not be the most efficient and productive way of ETL processing, we must remember the cost benefit analysis.

In a project where source and target databases are both the same (say Oracle), it makes perfect sense to carry out moderate ETL processing via PL/SQL alone as long as you get acceptable performance out of it.

The requirement of a full scale ETL tool most felt in a heterogeneous environment.

The main disadvantage of using traditional programming languages for ETL work is the performance. Programming languages tend process data in loops. For small data volume it is fine. However, looping technique is not very performing for large data sets. Yes, it is possible to tailor made your procedural language code to take advantage of parallelism (tormenting in C++ but bearable in PL/SQL) but in reality it is as involving as writing an ETL tool yourself and thus not really rewarding experience (unless you can market your tool).

In ETL tools, you don’t have to do anything (except keeping your data organized across partitions etc.) to gain performance advantage. They are designed to take parallelism etc. in their execution plan.

But in procedural language, you are of your own. One way to improve performance in PL/SQL (or similar) is to write cursors less often and use set operations more often. Most databases are designed to use best parallelism available when you use set operations.

Don’t forget cost of recruiting personnel in your chosen ETL tool. Depending on market condition, experts in one ETL tool might demand more compensation than other one. You also need to budget for organizing training for your existing staffs.

Most multinational companies can easily afford the best tool sets. However, for medium sized firms, budget is always a constraint. So, they are more prone to in-house development rather than buying tool set of the shelf.