Skip to main content
SpringerLink
Log in

Fitted Q-iteration and functional networks for ubiquitous recommender systems

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Ubiquitous recommender systems facilitate users on-location by personalized recommendations of items in the proximity via mobile devices. Due to a high variability of situations and preferences, an efficient resource processing is needed in order to assist the user in a proper way. In this paper, we consider a recommender system, able to learn preferences/habits of users through contextual information, such as location and time, using a new offline model-free approximate Q-iteration. Following the basic idea of Fitted Q-Iteration, the paper focuses on a computational scheme, based on functional networks, and that, unlike the well-known neural ones, does not require a large number of training samples. A preliminary case study, which deals with a shopping mall, is useful to show the approximation capabilities of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Acampora G, Gaeta M, Tomasiello S (2013) An extended functional network model and its application for a gas sensing system. Soft Comput 17(5):897–908

    Article  Google Scholar 

  • Adomavicius G, Tuzhilin A (2008) Context-aware recommender systems. In: Proceedings of the 2008 ACM conference on recommender systems, pp 335–336

  • Alonso-Betanzos A, Sanchez-Marono N, Carballal-Fortes FM, Suarez-Romero J, Perez-Sanchez B (2007) Classification of computer intrusions using functional networks: a comparative study. In: Proceedings of European symposium on artificial neural networks

  • Antos A, Munos R, Szepesvari C (2008) Fitted Q-iteration in continuous action-space MDPs. Adv Neural Inf Process Syst 20:9–16

    Google Scholar 

  • Baltrunas L (2008) Exploiting contextual information in recommender systems. In: Proceedings of the 2008 ACM conference on recommender systems, pp 295–298

  • Bruen M, Yang J (2005) Functional networks in real-time flood forecasting: a novel application. Adv Water Resour 28:899–909

    Article  Google Scholar 

  • Castillo E (1998) Functional networks. Neural Process Lett 7:151–159

    Article  MathSciNet  Google Scholar 

  • Castillo E, Cobo A, Gutiérrez JM, Pruneda E (1998) Working with differential, functional and difference equations using functional networks. Appl Math Model 23(2):89–107

    Article  MATH  Google Scholar 

  • Castillo E, Gutiérrez JM, Cobo A, Castillo C (2000) A minimax method for learning functional networks. Neural Process Lett 11(1):39–49

    Article  Google Scholar 

  • Castillo E, Iglesias A, Ruiz-Cobo R (2005) Funct Equ Appl Sci. Elsevier, Amsterdam

    Google Scholar 

  • David VK, Rajasekaran S (2009) Pattern recognition using neural and functional networks. In: Studies in computational intelligence, 160. Springer, Berlin

  • Dey AK, Abowd GD, Salber D (2001) A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Hum Comput Interact 16:97–166

    Article  Google Scholar 

  • El-Sebakhy EA (2008) New computational intelligence paradigm for estimating the software project effort. In: Proceedings of international conference on advanced information networking and applications (AINA)

  • El-Sebakhy EA, Hadi AS, Faisal KA (2007) Iterative least squares functional networks classifier. IEEE Trans Neural Netw 18(3):844–850

    Article  Google Scholar 

  • Ernst D, Geurts P, Wehenkel L (2005) Tree-based batch mode reinforcement learning. J Mach Learn Res 6:503–556

    MATH  MathSciNet  Google Scholar 

  • Fang B, Liao S, Xu K, Cheng H, Zhu C, Chen H (2012) A novel mobile recommender system for indoor shopping. Expert Syst Appl 39(15):11992–12000

    Article  Google Scholar 

  • Fano AE (1998) Shopper’s eye: using location-based filtering for a shopping agent in the physical world. In: AGENTS ’98 proceedings of the second international conference on autonomous agents, pp 416–421

  • Gaeta M, Loia V, Tomasiello S (2013) A generalized functional network for a classifier-quantifiers scheme in a gas-sensing system. Int J Intell Syst 28(10):988–1009

    Article  Google Scholar 

  • Gaeta M, Loia V, Miranda S, Tomasiello S (2016) Fitted Q—iteration by functional networks for control problems. Appl Math Model (to appear)

  • Gediminas A, Tuzhilin A (2005) Toward the next generation of recommender system: a survey of state-of the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):149–156

    Google Scholar 

  • Gordon GJ (1995) Online fitted reinforcement learning. In: VFA workshop at ML-95

  • Helmy T, Fatai A (2010) Hybrid computational intelligence models for porosity and permeability prediction of petroleum reservoirs. Int J Comput Intel Appl 9(4):313–337

    Article  Google Scholar 

  • Iglesias A, Arcay B, Cotos JM, Taboada JA, Dafonte C (2004) A comparison between functional networks and artificial neural networks for the prediction of fishing catches. Neural Comput Appl 13:24–31

    Article  Google Scholar 

  • Lacruz B, Perez-Palomares A, Pruneda RE (2006) Functional networks for classification and regression problems. In: Proceedings of international conference on mathematical and statistical modeling in Honor of Enrique Castillo

  • Mahmood T, Mujtaba G, Venturini A (2014) Dynamic personalization in conversational recommender systems. Inf Syst E-Bus Manag 12(2):213–238

    Article  Google Scholar 

  • Mahmood T, Ahmed SH, Mahmood S (2010) Comparing reward-based optimal behaviors in user-adapted recommender systems. In: Proceedings of 3rd IEEE international conference on computer science and information technology, vol 5, pp 332–336

  • Mettouris C, Papadopoulos GA (2014) Ubiquitous recommender systems. Computing 96(3):223–257

    Article  Google Scholar 

  • Neumann G, Peters J (2008) Fitted Q-iteration by advantage weighted regression. Adv Neural Inf Process Syst 21:1177–1184

    Google Scholar 

  • Ormoneit D, Sen S (2002) Kernel-based reinforcement learning. Mach Learn 49:161–178

    Article  MATH  Google Scholar 

  • Press WH, Teukolsky SA, Vetterling WT, Flannery BP (1992) Numerical recipes in C: the art of scientific computing, 2nd edn. Cambridge University Press, New York

    MATH  Google Scholar 

  • Pruneda RE, Lacruz B, Solares C (2005) A first approach to solve classification problems based on functional networks. In: Artificial neural networks: formal models and their applications. Lecture notes in computer science—ICANN, vol 3697, pp 313–318

  • Rajasekaran S, Thiruvenkatasamy K, Lee T-L (2006) Tidal level forecasting using functional and sequential learning neural networks. Appl Math Model 30:85–103

    Article  MATH  Google Scholar 

  • Resnick P, Varian HR (1997) Recommender systems. Commun ACM 40(3):56–58

    Article  Google Scholar 

  • Riedmiller M (2005) Neural fitted Q-iteration—first experiences with a data efficient neural reinforcement learning method. In: Machine learning: ECML 2005, Volume 3720 of the series Lecture Notes in Computer Science, pp 317–328

  • Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: Proceedings of the IEEE international conference on neural networks (ICNN)

  • Sanchez-Marono N, Alonso-Betanzos A (2007) Feature selection based on sensitivity analysis. In: Lecture notes in computer science—12th conference Spanish association for artificial intelligence and its associated conference on technology transfer on artificial intelligence (CAEPIA/TTIA), vol 4788, pp 239–248

  • Singh S, Litman D, Kearns M, Walker M (2002) Optimizing dialogue management with reinforcement learning: experiments with the NJFun system. J Artif Intell Res 16:105–133

    MATH  Google Scholar 

  • Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  • Taghipour N, Kardan A (2008) A hybrid web recommender system based on Q-learning, In: SAC ’08 proceedings of the 2008 ACM symposium on applied computing, pp 1164–1168

  • Timmer S, Riedmiller M (2007) Fitted Q–iteration with CMACs. In: Proceedings of IEEE international symposium on approximate dynamic programming and reinforcement learning (ADPRL)

  • Tomasiello S (2011) A functional network to predict fresh and hardened properties of selfcompacting concretes. Int J Numer Methods Biomed Eng 27(6):840–847

    Article  MATH  Google Scholar 

  • Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8:279–292

    MATH  Google Scholar 

  • Zhou Y, He D-X, Nong Z (2005) Application of functional networks tosolving classification problems. World Acad Sci EngTechnol 12:71–74

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, Ingegneria Elettrica e Matematica Applicata, University of Salerno, Via Giovanni Paolo II, 132, 84084, Fisciano, SA, Italy

    Matteo Gaeta & Stefania Tomasiello

  2. Dipartimento di Scienze Aziendali - Management and Innovation Systems, University of Salerno, Via Giovanni Paolo II, 132, 84084, Fisciano, SA, Italy

    Francesco Orciuoli

  3. CO.RI.SA., COnsorzio RIcerca Sistemi ad Agenti, University of Salerno, Via Giovanni Paolo II, 132, 84084, Fisciano, SA, Italy

    Luigi Rarità & Stefania Tomasiello

Authors
  1. Matteo Gaeta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesco Orciuoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luigi Rarità
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefania Tomasiello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefania Tomasiello.

Ethics declarations

Conflict of interest

All the authors declare that they have no conflict of interest.

Human and animals rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gaeta, M., Orciuoli, F., Rarità, L. et al. Fitted Q-iteration and functional networks for ubiquitous recommender systems. Soft Comput 21, 7067–7075 (2017). https://doi.org/10.1007/s00500-016-2248-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2248-1

Keywords

Search

Navigation