Data platform and Big Data – so what’s the trend is likely to be?

Can you predict the future?

Can you dictate your own future?

There is no possibility or simple answer to the first question, but there is an opportunity to answer the second question – shape/dictate your own future.

Not just with the future, just talk about new trends in data platform.Big Data is a buzz word in every domain and business. So how do you get into this stream of technologies and get a head-start.

Nothing is impossible when you have a zeal to learn….

There is no clear formulae to achieve success, but there is a process to start first step by learning  methodology and strategy to understand Data Platform and Big Data technologies with right tools.

Why I’m separating Data Platform and Big Data?

Data platform consist small data (DB) to very-large data (VLDB), whereas Big Data is a distribution system which is now a mainstream technology where many companies started their strategies (few of them got succeeded and few fall behind). In particular, keep a focus on real-time data analytics/analysis and machine-learning algorithms (have a look at Big Data category blogs here).

Using Big Data strategy one will be able to focus on smart data irrespective of size with the variety and volume. Unless you experiment with deployed technology and understanding it is essential to find stream, location, data-links, decision-makings and to-apply. IMHO, this will lead into Artificial Intelligence (AI) development with a mixture of data-anlytics & machine-learning. Let’s call this as a data science.

I have had a flurry of feedback for my posts in LinkedIn, so keeping focus on data science topic, open source technologies are throwing into a mix with mass & volume of how best a technology can help. So now is the time to look into tools ecosystem, services and vendors who have been built to tackle this subject by using many contributions with some amazing technoloy products. I’m not saying that use of technology will solve a problem, in-specific to data problems – real, virtual and physical. So this leads into specifics of highly available, scalable and performant data-applications to analyze data streams and adopt good decision-making for businesses.

The changes in data platform haven’t occured overnight, to talk about relational-data (BI) and data-warehousing (DW) has been mature enough now to adopt any kind of business challenge. By default any technology and product do have advantages/disadvantages with BI/DW relativitiy. Hence the shift in technologies with fundamental overview of how a business-problem can be solved, stepping into next-generation of data anlaytics (I’m not saying such and such product is good, just making a point on strategy).

To look into bigger picture about natural ability that you can start learning and progression within the technologies to solve business problems, Big Data is at a stage where analytics is added with datasets orginated from anywhere that is objective to the business need & challenge. Cloud computing needs a special mention here where the data accessibility is possible with few simple clicks (keeping governance & security controls in place). This leads into Open-Data concept which require a data dump, like how the software developers make best use of GitHub and CodePlex sites. A technology always requires skills and technique to master with few analytical and coding practice.

If you are going dive into Big Data then now is the time and opportunity to create data science division/practice in your organisation by formulating information & governance at certain level with right questions in place. No doubt that data surge happened with your day-to-day life and not to mention how organisations are coping to secure relevant messaing devices, get the feed, data exchange and accessibility to ensure trust & communication is secure, nobody likes encryption but it plays big part to get you successful without failling into wrong hands (phishing/scams), hence privacy is main pillar when you build a data platform.

Just to touchbase AI from previous point, now a days you can do a lot with your mobile device and think what else it can lend in future possibilities, Internet of Things (IoT) and Machine Learning (ML) is a mixture of deriving at this point to build insights, get value and extend data beyond the perimeter that can build a data platform.

Finally, I would like to close this post with a good impression (my search skills too) of how and what you could do with simple infographic below:



Taming Big Data (Source: IBM)

More later…..