Using Multimodal and Hyperlinked Representations of Knowledge as Academic Writing Aids
Pages 61 - 65
Abstract
Representing knowledge in written papers may be one of the biggest challenges that students face in higher education. This study analyzes how hypermedia structures can facilitate students' critical reflection on their papers by using multimodal resources. By converging academic writing, knowledge representation, and multimedia resources, we designed a hypermedia system that enables the visualization and representation of students' papers using text, images, audio, hyperlinks, and videos. To test the system, we conducted a pilot study in which we instructed 160 undergraduate students to write a paper in the following three-step exercise: First, students submitted an initial draft of their papers. Then, they used the system to translate the papers' content into different multimodal resources. Finally, they rewrote their papers with insights gained from the process. In a concluding survey, students reported that translating text to multimodal resources deepened their understanding of their papers' content and improved their topic organization.
References
[1]
MoizSAnsari,ZahidAliChannar,andAishaSyed.2012.Mobilephoneadoptionandappropriationamongtheyounggeneration.Procedia-SocialandBehavioralSciences41(2012),265--272.
[2]
MariusBrade,FlorianSchneider,AngelikaSalmen,andRainerGroh.2013.OntoS-ketch:TowardsDigitalSketchingAsaToolforCreatingandExtendingOntologiesforNon-Experts.InProceedingsofthe13thInternationalConferenceonKnowledgeManagementandKnowledgeTechnologies(i-Know'13).ACM,NewYork,NY,USA,Article9,8pages.https://doi.org/2494216
[3]
JohnBrookeetal.1996.SUS-Aquickanddirtyusabilityscale.Usabilityevaluationinindustry189,194(1996),4--7.
[4]
PeterBrusilovsky.1998.Methodsandtechniquesofadaptivehypermedia.InAdaptivehypertextandhypermedia.Springer,1--43.
[5]
PeterBrusilovsky,JohnEklund,andElmarSchwarz.1998.Web-basededucationforall:atoolfordevelopmentadaptivecourseware.ComputernetworksandISDNsystems30,1--7(1998),291--300.
[6]
RemoAslakBurkhard.2005.Towardsaframeworkandamodelforknowledgevisualization:Synergiesbetweeninformationandknowledgevisualization.InKnowledgeandinformationvisualizati on.Springer,238--255.
[7]
ManuelCastells.1996.TheRiseoftheNetworkSociety.TheInformationAge:Econ-omy,Society,andCultureVolumeI(InformationAgeSeries).London:Blackwell.
[8]
Paul DeBra, AdAerts, Bart Berden, Barendde Lange, Brendan Rousseau, Tomi Santic, David Smits, and Natalia Stash. 2003. AHA! The Adaptive Hypermedia Architecture. In Proceedings of the 14th ACM Conference on Hypertext and Hyper-media. ACM, New York, NY, USA,81--84.
[9]
MyrrhDomingo.2012.Linguisticlayering:Sociallanguagedevelopmentinthecontextofmultimodaldesignanddigitaltechnologies.Learning,MediaandTechnology37,2(2012),177--197.
[10]
MartinJEppler.2006.AComparisonbetweenConceptMaps,MindMaps,Con-ceptualDiagrams,andVisualMetaphorsasComplementaryToolsforKnowl-edgeConstructionandSharing.InformationVisualization5,3(2006),202--210.
[11]
GijsbertErkens,GellofKanselaar,MaaikePrangsma,andJosJaspers.2002.Usingtoolsandresourcesincomputersupportedcollaborativewriting.InProceedingsoftheConferenceonComputerSupportforCollaborativeLearning:FoundationsforaCSCLCommunity.InternationalSocietyoftheLearningSciences,389--398.
[12]
HaakonFasteandHonrayLin.2012.TheUntappedPromiseofDigitalMindMaps.InProceedingsoftheSIGCHIConferenceonHumanFactorsinComputingSystems(CHI'12).ACM,1017--1026.
[13]
JudeFransmanandRichardAndrews.2012.Rhetoricandthepoliticsofrepre-sentationandcommunicationinthedigitalage.
[14]
FrankHalaszandMayerSchwartz.1994.TheDexterHypertextReferenceModel.Commun.ACM37,2(Feb.1994),30--39.
[15]
CharlieHargood,FredCharles,andDavidE.Millard.2018.IntelligentGenerativeLocativeHyperstructure.InProceedingsofthe29thonHypertextandSocialMedia(HT'18).ACM,238--241.
[16]
KurtHornikandBettinaGrün.2011.topicmodels:AnRpackagefor?ttingtopicmodels.JournalofStatisticalSoftware40,13(2011),1--30.
[17]
AbhinavJain,NitinGupta,ShashankMujumdar,SameepMehta,andRishiMad-hok.2018.ContentDrivenEnrichmentofFormalTextUsingConceptDe?nitionsandApplications.InProceedingsofthe29thonHypertextandSocialMedia(HT'18).ACM,NewYork,NY,USA,96--100.
[18]
GuntherKress.2004.Readingimages:Multimodality,representationandnewmedia.InformationDesignJournal12,2(2004),110--119.
[19]
AndrejKrito andMáriaBieliková.2005.ImprovingAdaptationinWeb-basedEducationalHypermediabyMeansofKnowledgeDiscovery.InProceedingsofthe16thACMConferenceonHypertextandHypermedia.ACM,NewYork,NY,USA,184--192.
[20]
TamiLapidotandNoaRagonis.2013.SupportingHighSchoolComputerScienceTeachersinWritingAcademicPapers.InProceedingsofthe18thACMConferenceonInnovationandTechnologyinComputerScienceEducation(ITiCSE'13).ACM,325--325.
[21]
ChristineL.Liao.2007.CollaborationandDialogue:VideoAsanAgentinEx-tendingandEnrichingLearningandWriting.InACMSIGGRAPH2007EducatorsProgram.ACM,Article5.
[22]
TheresaMLillis.2002.Studentwriting:Access,regulation,desire.Routledge.
[23]
Z.Lu,M.Fan,Y.Wang,J.Zhao,M.Annett,andD.Wigdor.2019.InkPlanner:SupportingPrewritingviaIntelligentVisualDiagramming.IEEETransactionsonVisualizationandComputerGraphics25,1(Jan2019),277--287. {24}MaryMadden,AmandaLenhart,MaeveDuggan,SandraCortesi,andUrsGasser.2005.Teensandtechnology2013.PewInternet&AmericanLifeProject.
[24]
JenniferMcCormick.2011.Transmediationinthelanguageartsclassroom:Creatingcontextsforanalysisandambiguity.JournalofAdolescent&AdultLiteracy54,8(2011),579--587.
[25]
DeborahMcCutchen.2011.Fromnovicetoexpert:Implicationsoflanguageskillsandwriting-relevantknowledgeformemoryduringthedevelopmentofwritingskill.JournalofWritingResearch3,1(2011).
[26]
DavidE.MillardandCharlieHargood.2017.TireeTales:ACo-operativeInquiryintothePoeticsofLocation-BasedNarrative.InProceedingsofthe28thACMConferenceonHypertextandSocialMedia(HT'17).ACM,NewYork,NY,USA,15--24.
[27]
KathyMillsetal.2011.'I'mmakingitdi?erenttothebook':Transmediationinyoungchildren'smultimodalanddigitaltexts.AustralasianJournalofEarlyChildhood36,3(2011),56.
[28]
HugoMougard,MatthieuRiou,ColindelaHiguera,SolenQuiniou,andOlivierAubert.2015.ThePaperortheVideo:WhyChoose?.InProceedingsofthe24thInternationalConferenceonWorldWideWeb(WWW'15Companion).ACM,NewYork,NY,USA,1019--1022.
[29]
TedNelson.1982.Literarymachines.(1982).
[30]
IgnacioPerez-Messina,ClaudioGutierrez,andEduardoGraells-Garrido.2018.OrganicVisualizationofDocumentEvolution.In23rdInternationalConferenceonIntelligentUserInterfaces(IUI'18).ACM,NewYork,NY,USA,497--501.
[31]
ChristianRapp,OttoKruse,JenniferErlemann,andJakobOtt.2015.ThesisWriter:ASystemforSupportingAcademicWriting.InProceedingsofthe18thACMConferenceCompaniononComputerSupportedCooperativeWork&SocialComputing(CSCW'15Companion).ACM,NewYork,NY,USA,57--60.
[32]
JohnSadauskas,DaraghByrne,andRobertK.Atkinson.2015.MiningMemories:DesigningaPlatformtoSupportSocialMediaBasedWriting.InProceedingsofthe33rdAnnualACMConferenceonHumanFactorsinComputingSystems(CHI'15).ACM,NewYork,NY,USA,3691--3700.
[33]
JohnnySaldaña.2015.Thecodingmanualforqualitativeresearchers.Sage.
[34]
EhsanSherkat,SeyednaserNourashrafeddin,EvangelosE.Milios,andRosaneMinghim.2018.InteractiveDocumentClusteringRevisited:AVisualAnalyticsApproach.In23rdInternationalConferenceonIntelligentUserInterfaces(IUI'18).ACM,281--292.
[35]
MarjorieSiegel.1995.Morethanwords:Thegenerativepoweroftransmediationforlearning.CanadianJournalofEducation/Revuecanadienne del'education(1995),455--475.
[36]
ChristineDTippett.2016.Whatrecentresearchondiagramssuggestsaboutlearningwithratherthanlearningfromvisualrepresentationsinscience.Inter-nationalJournalofScienceEducation38,5(2016),725--746. {38}AstridWichmann.2012.PreparingWikisforEducationalSettings:TheRoleofDiscussionBoardUseinWiki-basedWriting.InProceedingsoftheEighthAnnualInternationalSymposiumonWikisandOpenCollaboration(WikiSym'12).ACM,NewYork,NY,USA,Article26,4pages.
[37]
KathrynZickuhr.2010.Generationsandtheirgadgets.PewInternet&AmericanLifeProject.
Index Terms
- Using Multimodal and Hyperlinked Representations of Knowledge as Academic Writing Aids
Recommendations
Multimodal spatial reference in mediated environments: users' preferences and the pragmatics of pointing and talking
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing SystemsThis paper describes the current results and future developments of a project on multimodal spatial reference in mediated environments. The database consists of video-recorded sessions, with 120 participants in three experimental designs, contrasting ...
Timing multimodal turn-taking for human-robot cooperation
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interactionIn human cooperation, the concurrent usage of multiple social modalities such as speech, gesture, and gaze results in robust and efficient communicative acts. Such multimodality in combination with reciprocal intentions supports fluent turn-taking. I ...
Multimodal interaction systems: information and time features
Multimodal interaction systems combine visual information (involving images, text, sketches and so on) with voice, gestures and other modalities to provide flexible and powerful dialogue approaches, enabling users to choose one or more of the multiple ...
Comments
Information & Contributors
Information
Published In
September 2019
326 pages
ISBN:9781450368858
DOI:10.1145/3342220
- General Chairs:
- Claus Atzenbeck,
- Jessica Rubart,
- Program Chair:
- David E. Millard
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 September 2019
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
- Comisión Nacional de Investigación Científica y Tecnológica
Conference
Acceptance Rates
HT '19 Paper Acceptance Rate 20 of 68 submissions, 29%;
Overall Acceptance Rate 378 of 1,158 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 162Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)2
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in