Your new post is loading...
Your new post is loading...
In the right hands and handled strategically, the massive amounts of information companies collect today can become a valuable new asset.
By 2015, nearly 3 billion people will be online, pushing the data created and shared to nearly 8 zettabytes. Is your network ready for the deluge? Exploring this infographic is the first step toward building a tangible plan and it may be the difference between reacting and prospering in Big Data’s shadow.
E-commerce site Etsy has grown to 25 million unique visitors and 1.1 billion page views per month, and it's generating the data volumes to match. Using tools such as Hadoop and Splunk, Etsy is turning terabytes of data per day into a better product.
Accel Partners has set aside $100 million to invest in start-ups that are trying to harness the power of big data, the firm will announce at the Hadoop World conference in New York this morning, aiming to consolidate its position as one of the earliest investors in these companies as the amount of data generated by companies and government agencies continues to grow.
"First, the doubling time of medical knowledge is currently estimated to be less than ten years and continues to decrease. Second, the volume of medical literature that a busy doctor needs to read to stay current keeps increasing and all too frequently different research reports present apparently contradictory evidence – frequently, no single treatment approach is suitable for all cases. Finally, as in all other aspects of our life, the amount of useful interpretable health data being collected has started to grow at an explosive rate. When you consider these factors, combined with the pressure to maintain quality while seeing increasing volumes of patients, it’s easy to understand why many doctors today feel that there has to be a different and more satisfying way to deliver the high quality care that they believe is possible." More: http://blogs.sas.com/content/hls/2011/10/04/healthcare-has-a-problem-big-data-the-law-of-seven/
Sybase has announced the newest version of its analytics database, Sybase IQ. This latest release, 15.4, is aimed at helping clients handle the challenges of mining big data. The vendor has expanded support for in-database analytics and IQ now integrates with Apache Hadoop, an open-source framework used for data analysis. The tool also has predictive model markup language (PMML) support and an expanded library of statistical and data mining algorithms. Dan Lahl, senior direct of product marketing at Sybase, and Neil McGovern, senior director of marketing for the financial services industry at Sybase, spoke to BST about the trends in the big data space and what the industry can expect from future Sybase releases.
Does big data have a roll in making the inbound call-center experience more enjoyable for everyone involved? Predictive analytics, says Deepak Advani of IBM, have a powerful potential role to play: "Basically, it’s a matter of putting analytics tools to work for customer service reps without them ever having to learn a thing about data analysis. In such a scenario, customer service isn’t left trying to deal with a potentially upset customer while simultaneously trying to find a resolution to his problem because the analytics system has done much of the legwork already." More: http://gigaom.com/cloud/big-data-making-even-call-centers-intelligent/
Former LinkedIn chief scientist DJ Patil shares advice on turning large-scale data into useful products.
Big Data company Cataphora believes there are six reasons *employees* would want their communications and other digital activity monitored: - If you’ve got nothing to hide, don’t hide anything. - There is a lot of good information to be garnered from monitoring communication - Even the best of us have bad days and even weeks. - Don’t sweat the small stuff. - Truth and lies (Discovery). - More and more employees work from home or remotely, requiring adjustments in management techniques. What do you think? Is extensive and comprehensive employee monitoring actually in the employee's best interest?
Comprehensive new book from O'Reilly on the privacy implications of the big data era: Much of what constitutes Big Data is information about us. Through our online activities, we leave an easy-to-follow trail of digital footprints that reveal who we are, what we buy, where we go, and much more. This eye-opening book explores the raging privacy debate over the use of personal data, with one undeniable conclusion: once data's been collected, we have absolutely no control over who uses it or how it is used. Personal data is the hottest commodity on the market today—truly more valuable than gold. We are the asset that every company, industry, non-profit, and government wants. Privacy and Big Data introduces you to the players in the personal data game, and explains the stark differences in how the U.S., Europe, and the rest of the world approach the privacy issue. You'll learn about: - Collectors: social networking titans that collect, share, and sell user data - Users: marketing organizations, government agencies, and many others - Data markets: companies that aggregate and sell datasets to anyone - Regulators: governments with one policy for commercial data use, and another for providing security
Line graphs and bar and pie charts and scatter plots all work well enough when there are just a few types of data or relationships to display. But in the ocean of big data they’re overwhelmed like dinghies in a storm. They’re limited to a flat world of two dimensions, an X and a Y, a vertical and a horizontal. There’s only so much they can show. They’re starting to look like finger paintings on a cave wall.
Are we prepared for what social media + big data will unleash? Good post from last spring at ZDNet still worth bubbling up: Today social media generates more information in a short period of time than was previously available in the entire world a few generations ago. Making sense of it and understanding what it means for your business will require all new technologies and techniques, including the emerging field of big data.
What are the boundaries to big data? Is the future bright for relational databases? Good questions to ask.
|
Johns Hopkins is taking a $1.2 million grant from the National Science Foundation to build a 100 gigabit per second network to shuttle data from the campus to other large computing centers at national labs and even at Google. The network will be capable of transferring an amount of data equivalent to 80 million file cabinets filled with text each day. More: http://gigaom.com/2011/11/08/for-science-big-data-is-the-microscope-of-the-21st-century/
Hadoop benefits elusive for typical enterprises; Informatica hopes to change that: "Informatica takes an important baby step towards bringing data transformation inside Hadoop, a development that will lower the barriers for enterprises wishing to tap the power of MapReduce processing in working with unstructured or variably structured big data. Informatica may not be the first vendor to develop tooling for processing and it certainly won’t be the last, but the release of its new HParser tool is an important step in making Hadoop a more civilized platform for the enterprise." More: http://ovum.com/2011/11/03/informatica-makes-first-step-to-transforming-data-inside-hadoop/
Kaggle is a platform where companies, researchers and governments can host competitions to help solve huge data-related problems. About 50 competitions have been run to date and members take just days to solve problems that have stumped scientists for years.
Big data computing is as much about speed as a surge in data, and that will bring sweeping changes in the computer industry. ... Not today or tomorrow perhaps, but big data portends striking changes for the computer industry, predicted [John E.] Kelly of I.B.M. “It’s going to break all of the technology we have and it’s where the next big arena of value is going to be in this industry,” he said. More: http://bits.blogs.nytimes.com/2011/10/31/big-data-speed-and-the-future-of-computing/
A "sponsored post" at Forbes from Praveen Asthana of Dell on "Big Data" and "Little Data". Ask the question: how exactly is big data different from "normal" sized data? "Big Data refers to huge amounts of data, hundreds of terabytes, even petabytes (1,000 terabytes) of information. It usually comes in the form of unstructured data and consists of data sets that may be unrelated to each other, such as bits from a variety of independent streams like social media, CRM, surveys, demographics, defects and so on. This is different from traditional data sets, which are often relational. Another key aspect of big data analytics is analytic velocity, or rapid, almost real-time analysis." More: http://www.forbes.com/sites/dell/2011/10/31/big-data-and-little-data/
A Big Data Manifesto from the Wikibon Community: "From storage and server technology that support Big Data processing to front-end data visualization tools that bring new insights alive for end-users, the emergence of Big Data also provides significant opportunities for hardware, software, and services vendors. Those vendors that aid enterprises in their transitions to Big Data practitioners, both in the form of identifying Big Data use cases that add business value and developing the technology and services to make Big Data a practical reality, will be the ones that thrive. Make no mistake: Big Data is the new definitive source of competitive advantage across all industries. Enterprises and technology vendors that dismiss Big Data as a passing fad do so at their peril and, in our opinion, will soon find themselves struggling to keep up with more foreword-thinking rivals. For those organizations that understand and embrace the new reality of Big Data, the possibilities for new innovation, improved agility, and increased profitability are nearly endless."
O'Reilly have a new book out about Privacy and Big Data. GigaOm asks the important question -- should there be legislative firewalls between Big Data and "Big Brother": We often laud big data when it's capturing and storing all sorts of new data types, but would the positive tone change if we we're talking about monitoring your every digital interaction while at work to discover questionable behavior?
Must read: The era of Big Data has begun. Computer scientists, physicists, economists, mathematicians, political scientists, bio-informaticists, sociologists, and many others are clamoring for access to the massive quantities of information produced by and about people, things, and their interactions. Diverse groups argue about the potential benefits and costs of analyzing information from Twitter, Google,Verizon, 23andMe, Facebook, Wikipedia, and every space where large groups of people leave digital traces and deposit data. Significant questions emerge. Will large-scale analysis of DNA help cure diseases? Or will it usher in a new wave of medical inequality? Will data analytics help make people’s access to information more efficient and effective? Or will it be used to track protesters in the streets of major cities? Will it transform how we study human communication and culture, or narrow the palette of research options and alter what ‘research’ means? Some or all of the above? This paper was presented at Oxford Internet Institute’s “A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society” on September 21, 2011
Gleaning insights from big data carries value across all aspects of baseball, from media outlets covering reactions to the game and players, to businesses marketing to fans, and to players and coaches themselves. Analyzing data to generate actionable insights in baseball affords major league teams better decision-making to create productive ball clubs year after year. For example, Oakland A's GM Billy Beane has become famous for the use of analytics, as seen in the best-selling book and motion picture, Moneyball.
We're in the age of the petabyte, exabyte and zettabyte. We're being overwhelmed with information, and we don't know what to do about it.
|