Keywords

1 Introduction

Web applications and platforms developed for image searching [1], media mining and social bookmarking [2] are rising in popularity and importance. During an information seeking process, users tend to revisit previously viewed images, data and locations, repeat searchers, and hence, they often have to memorize or manually create bookmarks for their interests. In order to facilitate the faster returns to their previous searching and views during image exploration process, the traditional bookmarks are becoming less used, leading to more novel algorithms that automatically generates image bookmarks on the basis of user interaction behavior. As per recent researches and previous ones also, implicit bookmarking mechanisms, is efficient for image and maps exploration, while their visual and interactive search [3] is quite intuitive to the users.

A traditional or manual bookmark is a saved shortcut, directing the user’s browser to a particular webpage. It stores page’s title, its URL and the favicon of corresponding page. Saving these bookmarks enable access to desired locations on the web. The idea of implicit bookmarking, which generates potential bookmarks on an auto basis for revisiting while the user is viewing required information, has been shown to be successfully in application domains like web browsing, image browsing, document reading, location mapping, social network analysis and data mining [4]. With automated bookmarking in these fields, media rich stories can be generated and shared across the platforms, including social media sites such as Facebook, Twitter, LinkedIn, Pinterest, Google Plus and more, giving a new dimension to social bookmarking [5].

1.1 Related Work

Previous research has explored methods for more effective and less repetitive bookmarking, with different techniques for supporting re-visitation, including user-operated manual operations and automated interfaces relied on recorded interaction history [6]. Explicit bookmarks interfaces and techniques need a series of possible user actions to save interesting information. For instance, in Adobe Reader, bookmark is added to bookmarks panel, in Google Maps, users add pushpin to My Places, click “save a map” link. In terms of document reading, Bookmark Scrollbar includes user-generated bookmarks on a regular scrollbar [7]. Now these traditional explicit bookmarks are faced with three basic downsides:

  • Manual marking of operations, which needs exertion of more cognitive efforts by the user

  • Interrupted interaction flows

  • Users must predict the sites they would be revisiting in future

These downsides have resulted in the need for new implicit bookmarking techniques and novel platforms. These upgrades have improvised user interfaces especially related to mining media techniques and tools as well as enhanced the efficiency of re-visitation. However, an element of truly intuitive interaction in these upgrades is still lacking [8]. These interfaces are deprived to show visiting history list, making users confused about the Back operational purpose and thus leading to their frustration due to inability to find the required item. Visited histories are usually hidden under other controls like those requiring the users to go through Recent Documents tab in the Word or a History tab given in a web browser. Current interfaces provide a higher degree of institution for showing visited history by using paths, annotations and images, yet, such tools are often limited to exploration tasks only in a discrete information scenario (Fig. 1).

Fig. 1.
figure 1

Picturemarks is a proof-of-concept system that leverages the power of the crowd to annotate, share and augment works with the aim to displace more meaningful interaction and discoveries. (a) A collection of various film noir posters from the web platform Amazon.com and (b) a collection of Homeless Paintings of the Italian Renaissance imported using a JSON file provided by a museum. (c) Another collection of images related to a PhD dissertation through custom image import and (d) a collection of surfing themed images imported from the web platform Giphy.com. All of the images in this paper are unedited; showing the real-time, working prototype.

Full size image

1.2 Task Model and Algorithm

The power of new algorithms for visualizing a plethora om images using implicit bookmarks for enhanced media mining and digital storytelling [9], lies in its feature to determine useful image sources users use or would like revisit in the future [10]. Here, we have included a proposed conceptual model for information seeking interactions with a large collection of images [11]. Picturemarks is a novel addition to the list of one of the most useful and logical sites for generating implicit bookmarks that users would re-visit. This is a bookmark in the dynamic and continuous information space that includes keyword search [12], viewport locations, zoom level, saving image collections, annotation, spotlight mode, sharing, commenting and import interfaces [13]. A re-visit is defined as the visit to the last viewport with a similar search i.e. re-doing a query. On this user interface model, our platform tends to exhibit three heuristic opportunities in mining media and digital storytelling, based on its implicit bookmarks generation:

  • To develop a web platform when a user starts a new search (i.e. showing newly requested search on the web)

  • Implicit bookmarking occurrence at the time user visits various places, giving an idea during multi-layered exploration. Long distance panning and zooming, saving and sharing of images

  • Generation during particular pauses, which is a time that shows that the user is more interested in browsing a particular location in the image collection.

2 The Platform

Our vision for a fully developed Picturemarks is a platform that explores, evaluates and makes arguments about image collections and images they contain. The platform leverages the collections of images to generate digital storytelling by providing users necessary tools including item level annotations to highly detailed, repository-wide visualizations [14]. These effective tools enable users to integrate image objects with communities to which they belong, so as to enable dialogue among one another. Digital storytelling thus works here in this way (Fig. 2).

Fig. 2.
figure 2

(Source: Picturemarks official website).

The layout of the Picturemarks canvas

Full size image

2.1 A Vast World of Visualization Tools

The platform shouldn’t be taken as an online exhibition platform. It is fundamentally a system where users pursue and share collection-based images along with a digital object identifier in a nimby, intuitive and iterative manner. The users can browse through a large number of images based on implicit bookmarking by using an increasing library of visualization tools for generating dynamic data portraits of the image collections.

In addition, annotation, image tagging, image re-organizing to demonstrate juxtapositions and relationships, are also enabled to provide new curatorial practice [15]. Users can then assemble these records in images and visualizations trays for sharing and working collaboratively with the people in their social circles. The custom collection trays containing assembled records are transformed into such published spotlights that greatly help unlocking those stories and augments that are bound up in those image collections or records.

2.2 The Exemplified Affordances

  • Navigating fluidly from the macro to the micro: For the art historian to look beyond the image object-level and the social scientist to look more intimately than the image collection-level. Linking among these macro and micro tiers.

  • Shadow collections: Museum visitors should be able to interact with image collection pieces not in the gallery or how the pieces that are physically present network to related works.

  • Custom collections: Being able to pull images to construct arguments and tell stories about a series of images and collections.

  • Demystifying large-scale data with visualizations: How to make sense of and see patterns in macro-level data.

  • Enhancing the metadata: Being able to annotate image objects in a way that can enhance and enrich the existing database, either through inherent expertise of users or crowd-based identifications and gamified task-assigning or search. Palettes of tags. Annotations beyond text i.e. with networked image objects, with audio, with video, etc.

  • Social dialogue: Being able to interact with current users and those of the past. Having dialogues around images and collections either within groups (i.e. classes) or in a broader public sense. View an image through another classmate or a friend’s “eyes” with their annotations active. Dialogue between singular image with the network of images.

  • Dynamic program of things: Being able to create visual syllabi for programs from database entries and images—being able to assign, organize, corral, and discuss program assignments within that framework. Linking from program assignments to user contributions and curator commentary.

3 User Interface

The first step is based on finding and following the desired or favorite image collections from a popular social media platform like Pinterest, Giphy, Amazon, StackOverflow or Instagram. As an alternative, users can import the collection by using “Import your own collection” option. However, to use this option required import API, image files and/or JSON files and offsite hosting and limited amount of metadata is included (Fig. 3).

Fig. 3.
figure 3

A screenshot of collecting files.

Full size image

In this step, the users can browse, filter, search and generate visualizations from any saved or followed image collection. Annotations, tagging and editing in metadata is also done in this step. The user can then save finalized individual images or collections of images or visualizations to his custom image collection. Categorization of saved records is also possible by using as many custom image collections as required (Fig. 4).

Fig. 4.
figure 4

A screenshot of exploring image collections.

Full size image

Once image collection(s) is explored, user can use the share function to work with members for collecting and ordering images, in real-time.

In this step, the user starts visualizing interrelationship by pulling information directly from the content in imported image collections or saving in the custom image collection (Fig. 5).

Fig. 5.
figure 5

A screenshot of custom image collections.

Full size image

Here, the user can also publish, share and export his/her spotlights. The user is able to edit images or collections, custom them and also edit spotlights. The user can join or initiate users for sharing and collaborating on work in particular image collections (Fig. 6).

Fig. 6.
figure 6

A screenshot of spotlight mode.

Full size image

This step gives users two ways to search annotations and for tagging the images (Fig. 7).

Fig. 7.
figure 7

A screenshot of search mode.

Full size image

  • The user can create annotations on the image spotlight page straightaway, where public can choose to view annotations or hide them in the Spotlight mode.

  • If annotations or tags are specific to certain groups or curators and an argument in development, tags and annotations can be made directly in the spotlight editor.

4 Evaluation

The goal of our study is to evaluate the usefulness of Picturemarks for supporting exploration of large sets of images with implicit bookmarks during users’ every day browsing interactions. The objective of wider access to data and images is to make them available to anyone, anytime, anywhere and for justified purposes. The enormous amount of image data that is accessible publicly is likely to be related with numerous sources irrespective of the publication date, impact factor and country of production. In view of the need to enhance user’s access to image data, while avoiding repeated search and revisited searching, a global data citing practice has now included digital object identifier (DOI) protocol in its eco system. Data citations in image firms means the way of giving a reference to image and relevant data in the same way as researches generally give a bibliographic reference to some printed resource. The DOI-based image bookmarking, annotation, curation, collaborative tagging [16, 17] that allow users to actively engage in knowledge creation [18]. The sharing enables exploration of huge data sets at a time in a globalized, standardized and widely applied manner, leading to enhanced form of community-based storytelling.

5 Results

In the current version of the platform, a large number of images are displayed and since the browser can load just two remote resources at the same time, a need might arise sometimes to load an urgent image before huge queue of images. In order to manage this, the platform includes an image loader for managing these queues, defined by the cursor movement onto image grid. If one collection of images is to be placed onto the grip, the system develops a series of these image objects for processing. This feature informs what image object must be processed next and it is based on the upcoming algorithm:

  • It is to be checked if urgent queue holds an image and loads if value is found to be true

  • It is to be noted that when this queue doesn’t contain any image object, it takes the image object from queue based on the nearest distance from cursor so as to make it easier for the user to load selective images in collections

This has been developed in case the user wants to open of the images in spotlight view and not enabling the image to get loaded until the whole collection is loaded. In a more specific manner, the user puts a cursor over an image containing a specific value and sort the value in the images queue. At present, the platform also supports ascending, descending, alphabetical and chronological sorting of the image collections.

6 Discussion

Picturemarks is the platform for fast learning of image objects and management of image collection during the mining media and digital storytelling process for implicit bookmarking. It is leveraged upon the unique interface of use interaction, viewing thousands of images side by side, sorting huge collections in a faster manner, developing and sharing thematic trays of images, mapping annotations and tags based around specific areas of relevance. Users can keep saving the selected images in the custom collection mode for producing stories and social dialogue.

The platform lets users annotate and tag the images. Annotations expected to gain high public’s interests, can be developed by the user on his image spotlight. This is the place where annotations are visible to public. As an option, if annotation doesn’t seem interesting to the users, they can hide them in their spotlight mode. Since our system is aimed for curators and influences, it is seen that many of the uses’ annotations and tags are directed towards certain groups only. Here, users can use the spotlight editor to keep the audience specific. The interface also enabled editing images, or collections of images, making them customized with edited spotlights. Tracking of all changes is available.

User starts sharing and collaborating the image collections, once initiated by the user. Specific image collections let user see history feed, updated annotations, and other updates via notifications. Commenting is also enabled for leading towards effective storytelling practices.

7 Conclusion and Future Work

In this paper we presented a proof-of-concept system for implicit bookmarking for image information-oriented tasks, where the data space is dynamic, continuous and multi-scale. The platform that explores, evaluates and makes arguments about large image collections and images they contain. The platform leverages the collections of images to generate digital storytelling by providing users necessary tools including item level annotations to highly detailed, repository-wide visualizations. These effective tools enable users to integrate image objects with communities to which they belong. The users, mostly curators and influencers, are actively seeking platforms for their web-based image searches, and they are highly satisfied with the usefulness of this implicit image bookmarking platform and its intuitive interaction and visualization, which ensures a fluid and effective image exploration practices. While we explored several points in the design space, there are many more left to be explored, for example its algorithm usability and interface improvisation. Furthermore, the system can further be adapted to higher compatibility with mobile versions in the wild. At present, the platform supports users’ short-term history of searching and interests, with implicitly generating bookmarks and visualization of source-name labels. Long term-history based additional features can be worked upon in future.