Organic streams a unified framework for personal big data integration and organization towards social sharing and individualized sustainable use

This chapter describes a unified framework for dynamically integrating and meaningfully organizing personal and social Big Data. With the rapid development of emerging computing paradigms, we have been continuously experiencing a change in work, life, playing, and learning in the highly developed information society, which is a kind of seamless integration of the real physical world and cyber digital space. More and more people have been accustomed to sharing their personal contents across the social networks due to the high accessibility of social media along with the increasingly widespread adoption of wireless mobile computing devices. User-generated information has spread more widely and quickly and provided people with opportunities to obtain more knowledge and information than ever before, which leads to an explosive increase of data scale, containing big potential value for individual, business, domestic, and national economy development. Thus, it has become an increasingly important issue to sustainably manage and utilize personal Big Data, in order to mine useful insight and real value to better support information seeking and knowledge discovery. To deal with this situation in the Big Data era, a unified approach to aggregation and integration of personal Big Data from life logs in accordance with individual needs is considered essential and effective, which can benefit the sustainable information sharing and utilization process in the social networking environment. In this chapter, a new concept of organic stream, which is designed as a flexibly extensible data carrier, is introduced and defined to provide a simple but efficient means to formulate, organize, and represent personal Big Data. As an abstract data type, organic streams can be regarded as a logic metaphor, which aims to meaningfully process the raw stream data into an associatively and methodically organized form, but no concrete implementation for physical data structure and stor­age is defined. Under the conceptual model of organic streams, a heuristic method is proposed and applied to extract diversified individual needs from the tremendous amount of social stream data through social media. And an integrated mechanism is developed to aggregate and integrate the relevant data together based on individual needs in a meaningful way, in which personal data can be physically stored and distributed in private personal clouds and logically represented and processed by a set of newly introduced metaphors named heuristic stone, associative drop, and associative ripple the architecture of the system with the foundational modules is described, and the prototype implementation with the experiment’s result is presented to demonstrate the usability and effectiveness of the framework and system.