Utku Sirin
Skip Abstract Section
Abstract
Numerous applications today rely on artificial intelligence over images. Image AI is, however, extremely expensive. In particular, the inference cost of image AI dominates the end-to-end cost. We observe that the image storage format lies at the root of the problem. Images today are predominantly stored in JPEG format. JPEG is a storage format designed for the human eye; it maximally compresses images without distorting the components of an image that are visible to the human eye. However, our observation is that during image AI, images are "seen'' by algorithms, not humans. In addition, every AI application is different regarding which data components of the images are the most relevant.
We present the Image Calculator, a self-designing image storage format that adapts to the given AI task, i.e., the specific neural network, the dataset, and the applications' specific accuracy, inference time, and storage requirements. Contrary to the state-of-the-art, the Image Calculator does not use a fixed storage format like JPEG. Instead, it designs and constructs a new storage format tailored to the context. It does so by constructing a massive design space of candidate storage formats from first principles, within which it searches efficiently using composite performance models (inference time, accuracy, storage). This way, it leverages the given AI task's unique characteristics to compress the data maximally. We evaluate the Image Calculator across a diverse set of data, image analysis tasks, AI models, and hardware. We show that the Image Calculator can generate image storage formats that reduce inference time by up to 14.2x and storage by up to 8.2x with a minimal loss in accuracy or gain, compared to JPEG and its state-of-the-art variants.
- Muhammad Adnan, Yassaman Ebrahimzadeh Maboud, Divya Mahajan, and Prashant J. Nair. 2021. Accelerating Recommendation System Training by Leveraging Popular Choices. Proc. VLDB Endow., Vol. 15, 1 (2021), 127--140.Google Scholar
Digital Library
- Gbeminiyi Ajayi. 2018. Multi-class Weather Dataset for Image Classification. Mendeley Data, Vol. 1 (2018). https://doi.org/10.17632/4drtyfjtfy.1Google ScholarCross Ref
- Baba Fakruddin Ali B H and Prakash Ramachandran. 2022. Compressive Domain Deep CNN for Image Classification and Performance Improvement Using Genetic Algorithm-Based Sensing Mask Learning. Applied Sciences, Vol. 12, 14 (2022).Google Scholar
- AWS. 2019. Amazon EC2 Update -- Inf1 Instances with AWS Inferentia Chips for High Performance Cost-Effective Inferencing. https://aws.amazon.com/blogs/aws/amazon-ec2-update-inf1-instances-with-aws-inferentia-chips-for-high-performance-cost-effective-inferencing/ Accessed on May 16, 2023.Google Scholar
- Yuanchao Bai, Xu Yang, Xianming Liu, Junjun Jiang, Yaowei Wang, Xiangyang Ji, and Wen Gao. 2022. Towards End-to-End Image Compression and Analysis with Transformers. In AAAI. 104--112.Google Scholar
- Favyen Bastani, Songtao He, Arjun Balasingam, Karthik Gopalakrishnan, Mohammad Alizadeh, Hari Balakrishnan, Michael Cafarella, Tim Kraska, and Sam Madden. 2020a. MIRIS: Fast Object Track Queries in Video. In SIGMOD. 1907--1921.Google Scholar
- Favyen Bastani, Songtao He, Arjun Balasingam, Karthik Gopalakrishnan, Mohammad Alizadeh, Hari Balakrishnan, Michael Cafarella, Tim Kraska, and Sam Madden. 2020b. MIRIS: Fast Object Track Queries in Video. In SIGMOD. 1907--1921.Google Scholar
- Vasudev Bhaskaran and Konstantinos Konstantinides. 1995. Image and Video Compression Standards. Springer.Google Scholar
- Imene Bouderbal, Abdenour Amamra, M. El-Arbi Djebbar, and Mohamed Akrem Benatia. 2022. Towards SSD Accelerating for Embedded Environments: A Compressive Sensing Based Approach. Journal of Real-Time Image Processing, Vol. 19, 6 (2022), 1199--1210.Google ScholarDigital Library
- Matthew Butrovich, Wan Shen Lim, Lin Ma, John Rollinson, William Zhang, Yu Xia, and Andrew Pavlo. 2022. Tastes Great! Less Filling! High Performance and Accurate Training Data Collection for Self-Driving Database Management Systems. In SIGMOD. 617--630.Google Scholar
- Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2020. Once-for-All: Train One Network and Specialize it for Efficient Deployment. In ICLR.Google Scholar
- Han Cai, Ji Lin, Yujun Lin, Zhijian Liu, Haotian Tang, Hanrui Wang, Ligeng Zhu, and Song Han. 2022. Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications. ACM Transactions Design Automation of Electronic Systems, Vol. 27, 3 (2022).Google ScholarDigital Library
- Han Cai, Ligeng Zhu, and Song Han. 2019. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In ICLR.Google Scholar
- Jiashen Cao, Karan Sarkar, Ramyad Hadidi, Joy Arulraj, and Hyesoon Kim. 2022. FiGO: Fine-Grained Query Optimization in Video Analytics. In SIGMOD. 559--572.Google Scholar
- Lahiru D. Chamain, Siyu Qi, and Zhi Ding. 2021a. An End-to-End Learning Architecture for Efficient Image Encoding and Deep Learning. In 29th European Signal Processing Conference (EUSIPCO). 691--695.Google Scholar
- Lahiru D. Chamain, Siyu Qi, and Zhi Ding. 2022. End-to-End Image Classification and Compression With Variational Autoencoders. IEEE Internet of Things Journal, Vol. 9, 21 (2022), 21916--21931.Google ScholarCross Ref
- Lahiru D. Chamain, Fabien Racapé, Jean Bégaint, Akshay Pushparaja, and Simon Feltman. 2021b. End-to-End Optimized Image Compression for Machines, A Study. In 31st Data Compression Conference (DCC). 163--172.Google Scholar
- Subarna Chatterjee, Meena Jagadeesan, Wilson Qin, and Stratos Idreos. 2022. Cosine: A Cloud-Cost Optimized Self-Designing Key-Value Storage Engine. Proc. VLDB Endow., Vol. 15, 1 (2022), 112--126.Google ScholarDigital Library
- Subarna Chatterjee, Mark Pekala, Lev Kruglyak, and Stratos Idreos. 2024. Limousine: Blending Learned and Classical Indexes to Self-Design Larger-than-Memory Cloud Storage Engines. Proc. ACM Manag. Data (2024).Google ScholarDigital Library
- Sien Chen, Jian Jin, Lili Meng, Weisi Lin, Zhuo Chen, Tsui-Shan Chang, Zhengguang Li, and Huaxiang Zhang. 2021. A New Image Codec Paradigm for Human and Machine Uses. CoRR, Vol. abs/2112.10071 (2021).Google Scholar
- Hyomin Choi and Ivan V. Bajic. 2018. Deep Feature Compression for Collaborative Object Detection. In 25th IEEE International Conference on Image Processing (ICIP). 3743--3747.Google Scholar
- Hyomin Choi and Ivan V. Bajic. 2021. Latent-Space Scalability for Multi-Task Collaborative Intelligence. In 28th IEEE International Conference on Image Processing (ICIP). 3562--3566.Google Scholar
- Hyomin Choi and Ivan V. Bajic. 2022. Scalable Image Coding for Humans and Machines. IEEE Transactions on Image Processing, Vol. 31, 1 (2022), 2739--2754.Google ScholarDigital Library
- Hyomin Choi, Robert A. Cohen, and Ivan V. Bajic. 2020. Back-And-Forth Prediction for Deep Tensor Compression. In 45th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 4467--4471.Google Scholar
- Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. 2018. PACT: Parameterized Clipping Activation for Quantized Neural Networks. CoRR, Vol. abs/1805.06085 (2018).Google Scholar
- Pramod Chunduri, Jaeho Bang, Yao Lu, and Joy Arulraj. 2022. Zeus: Efficiently Localizing Actions in Videos Using Reinforcement Learning. In SIGMOD. 545--558.Google Scholar
- Felipe Codevilla, Jean Gabriel Simard, Ross Goroshin, and Chris Pal. 2021. Learned Image Compression for Machine Perception. CoRR, Vol. abs/2111.02249 (2021).Google Scholar
- Nilaksh Das, Sanya Chaba, Renzhi Wu, Sakshi Gandhi, Duen Horng Chau, and Xu Chu. 2020. GOGGLES: Automatic Image Labeling with Affinity Coding. In SIGMOD. 1717--1732.Google Scholar
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A Large-scale Hierarchical Image Database. In CVPR. 248--255.Google Scholar
- Yingpeng Deng and Lina J. Karam. 2021. Learning-based Compression for Material and Texture Recognition. CoRR, Vol. abs/2104.10065 (2021).Google Scholar
- Radosvet Desislavov, Fernando Martínez-Plumed, and José Hernández-Orallo. 2023. Trends in AI Inference Energy Consumption: Beyond the Performance-vs-Parameter Laws of Deep Learning. Sustainable Computing: Informatics and Systems, Vol. 38 (2023), 100--857.Google ScholarCross Ref
- Jialin Ding, Vikram Nathan, Mohammad Alizadeh, and Tim Kraska. 2020. Tsunami: A Learned Multi-Dimensional Index for Correlated Data and Skewed Workloads. Proc. VLDB Endow., Vol. 14, 2 (2020), 74--86.Google ScholarDigital Library
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR.Google Scholar
- Iddo Drori and Joaquin Vanschoren. 2021. AAAI 2021 Meta Learning Tutorial. https://sites.google.com/mit.edu/aaai2021metalearningtutorial Accessed on May 16, 2023.Google Scholar
- Lingyu Du and Guohao Lan. 2022. FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning. CoRR, Vol. abs/2209.06692 (2022).Google Scholar
- Max Ehrlich. 2020. TorchJPEG. https://torchjpeg.readthedocs.io/en/latest/ Accessed on May 16, 2023.Google Scholar
- Max Ehrlich, Larry Davis, Ser-Nam Lim, and Abhinav Shrivastava. 2020. Quantization Guided JPEG Artifact Correction. ECCV (2020).Google Scholar
- Jingzhi Fang, Yanyan Shen, Yue Wang, and Lei Chen. 2021. ETO: Accelerating Optimization of DNN Operators by High-Performance Tensor Program Reuse. Proc. VLDB Endow., Vol. 15, 2 (2021), 183--195.Google ScholarDigital Library
- Forbes. 2019. Google Cloud Doubles Down On NVIDIA GPUs For Inference. https://www.forbes.com/sites/moorinsights/2019/05/09/google-cloud-doubles-down-on-nvidia-gpus-for-inference/ Accessed on May 16, 2023.Google Scholar
- Dan Fu and Gabriel Guimaraes. 2016. Using Compression to Speed Up Image Classification in Artificial Neural Networks. (2016). https://www.danfu.org/files/CompressionImageClassification.pdfGoogle Scholar
- Fangeheng Fu, Jiawei Jiang, Yingxia Shao, and Bin Cui. 2019. An Experimental Evaluation of Large Scale GBDT Systems. Proc. VLDB Endow., Vol. 12, 11 (2019), 1357--1370.Google ScholarDigital Library
- Fangcheng Fu, Xupeng Miao, Jiawei Jiang, Huanran Xue, and Bin Cui. 2022. Towards Communication-Efficient Vertical Federated Learning Training via Cache-Enabled Local Updates. Proc. VLDB Endow., Vol. 15, 10 (2022), 2111--2120.Google ScholarDigital Library
- Shay Gershtein, Tova Milo, Slava Novgorodov, and Kathy Razmadze. 2022. Classifier Construction Under Budget Constraints. In SIGMOD. 1160--1174.Google Scholar
- Stefan Grafberger, Paul Groth, and Sebastian Schelter. 2023. Automating and Optimizing Data-Centric What-If Analyses on Native Machine Learning Pipelines. Proc. ACM Manag. Data, Vol. 1, 2 (2023).Google ScholarDigital Library
- Lionel Gueguen, Alex Sergeev, Ben Kadlec, Rosanne Liu, and Jason Yosinski. 2018. Faster Neural Networks Straight from JPEG. In NeurIPS. 3937--3948.Google Scholar
- Isabelle Guyon, Jan N. van Rijn, Sé bastien Treguer, and Joaquin Vanschoren (Eds.). 2021. AAAI Workshop on Meta-Learning and MetaDL Challenge, MetaDL@AAAI 2021, virtual, February 9, 2021. Proceedings of Machine Learning Research, Vol. 140. PMLR. https://proceedings.mlr.press/v140/Google Scholar
- Song Han, Huizi Mao, and William J. Dally. 2016. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding. In ICLR.Google Scholar
- Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both Weights and Connections for Efficient Neural Network. In NuerIPS. 1135--1143.Google Scholar
- Brandon Haynes, Maureen Daum, Dong He, Amrita Mazumdar, Magdalena Balazinska, Alvin Cheung, and Luis Ceze. 2021. VSS: A Storage System for Video Analytics. In SIGMOD. 685--696.Google Scholar
- Kaiming He, Georgia Gkioxari, Piotr Dollá r, and Ross B. Girshick. 2017a. Mask R-CNN. In ICCV. 2980--2988.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770--778.Google Scholar
- Wenjia He, Michael R. Anderson, Maxwell Strome, and Michael Cafarella. 2020. A Method for Optimizing Opaque Filter Queries. In SIGMOD. 1257--1272.Google Scholar
- Wenjia He and Michael Cafarella. 2022. Controlled Intentional Degradation in Analytical Video Systems. In SIGMOD. 2105--2119.Google Scholar
- Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. 2018. AMC: AutoML for Model Compression and Acceleration on Mobile Devices. In ECCV, Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). 815--832.Google Scholar
- Yihui He, Xiangyu Zhang, and Jian Sun. 2017b. Channel Pruning for Accelerating Very Deep Neural Networks. In ICCV. 1398--1406.Google Scholar
- Andrew Howard, Ruoming Pang, Hartwig Adam, Quoc V. Le, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, and Yukun Zhu. 2019. Searching for MobileNetV3. In ICCV. 1314--1324.Google Scholar
- HPCwire. 2019. AWS to Offer Nvidia's T4 GPUs for AI Inferencing. https://www.hpcwire.com/2019/03/19/aws-upgrades-its-gpu-backed-ai-inference-platform/ Accessed on May 16, 2023.Google Scholar
- Bo Hu, Peizhen Guo, and Wenjun Hu. 2022. Video-Zilla: An Indexing Layer for Large-Scale Video Analytics. In SIGMOD. 1905--1919.Google Scholar
- Yueyu Hu, Shuai Yang, Wenhan Yang, Ling-Yu Duan, and Jiaying Liu. 2020. Towards Coding For Human And Machine Vision: A Scalable Image Coding Approach. In 21st IEEE International Conference on Multimedia and Expo (ICME). 1--6.Google Scholar
- Yueyu Hu, Wenhan Yang, Haofeng Huang, and Jiaying Liu. 2021. Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective. CoRR, Vol. abs/2106.08512 (2021).Google Scholar
- Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized Neural Networks. In NeurIPS.Google Scholar
- Stratos Idreos, Niv Dayan, Wilson Qin, Mali Akmanalp, Sophie Hilgard, Andrew Ross, James Lennon, Varun Jain, Harshita Gupta, David Li, and Zichen Zhu. 2019a. Design Continuums and the Path Toward Self-Designing Key-Value Stores that Know and Learn. In CIDR.Google Scholar
- Stratos Idreos and Tim Kraska. 2019. From Auto-Tuning One Size Fits All to Self-Designed and Learned Data-Intensive Systems. In SIGMOD. 2054--2059.Google Scholar
- Stratos Idreos, Kostas Zoumpatianos, Manos Athanassoulis, Niv Dayan, Brian Hentschel, Michael S. Kester, Demi Guo, Lukas M. Maas, Wilson Qin, Abdul Wasay, and Yiyou Sun. 2018a. The Periodic Table of Data Structures. IEEE Data Eng. Bull., Vol. 41, 3 (2018), 64--75.Google Scholar
- Stratos Idreos, Kostas Zoumpatianos, Subarna Chatterjee, Wilson Qin, Abdul Wasay, Brian Hentschel, Mike S. Kester, Niv Dayan, Demi Guo, Minseo Kang, and Yiyou Sun. 2019b. Learning Data Structure Alchemy. IEEE Data Eng. Bull., Vol. 42, 2 (2019), 47--58.Google Scholar
- Stratos Idreos, Konstantinos Zoumpatianos, Brian Hentschel, Michael S. Kester, and Demi Guo. 2018b. The Data Calculator: Data Structure Design and Cost Synthesis From First Principles, and Learned Cost Models. In SIGMOD. 535--550.Google Scholar
- iMerit. 2021. Top 13 Machine Learning Image Classification Datasets. https://imerit.net/blog/top-13-machine-learning-image-classification-datasets-all-pbm/ Accessed on Jan. 02, 2023.Google Scholar
- Alexander Isenko, Ruben Mayer, Jeffrey Jedele, and Hans-Arno Jacobsen. 2022. Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines. In SIGMOD. 1825--1839.Google Scholar
- Jiawei Jiang, Shaoduo Gan, Yue Liu, Fanlin Wang, Gustavo Alonso, Ana Klimovic, Ankit Singla, Wentao Wu, and Ce Zhang. 2021. Towards Demystifying Serverless Machine Learning Training. In SIGMOD. 857--871.Google Scholar
- Wenbin Jiao, Xuemin Cheng, Yao Hu, Qun Hao, and Hongsheng Bi. 2022. Image Recognition Based on Compressive Imaging and Optimal Feature Selection. IEEE Photonics Journal, Vol. 14, 2 (2022), 1--12.Google ScholarCross Ref
- Sian Jin, Chengming Zhang, Xintong Jiang, Yunhe Feng, Hui Guan, Guanpeng Li, Shuaiwen Leon Song, and Dingwen Tao. 2021. COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression. Proc. VLDB Endow., Vol. 15, 4 (2021), 886--899.Google ScholarDigital Library
- Kaggle. 2023 a. Cigarette Filter Detection Task. https://www.kaggle.com/datasets/estebanpacanchique/cigarette-butt Accessed on Sept. 20, 2023.Google Scholar
- Kaggle. 2023 b. Micro-controller Detection and Segmentation Task. https://www.kaggle.com/datasets/tannergi/microcontroller-segmentation Accessed on Sept. 20, 2023.Google Scholar
- Daniel Kang, Peter Bailis, and Matei Zaharia. 2019. BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics. Proc. VLDB Endow., Vol. 13, 4 (2019), 533--546.Google ScholarDigital Library
- Daniel Kang, Ankit Mathur, Teja Veeramacheneni, Peter Bailis, and Matei Zaharia. 2020. Jointly Optimizing Preprocessing and Inference for DNN-Based Visual Analytics. Proc. VLDB Endow., Vol. 14, 2 (2020), 87--100.Google ScholarDigital Library
- Barrie Kersbergen, Olivier Sprangers, and Sebastian Schelter. 2022. Serenade - Low-Latency Session-Based Recommendation in e-Commerce at Scale. In SIGMOD. 150--159.Google Scholar
- Michael S. Kester, Manos Athanassoulis, and Stratos Idreos. 2017. Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe?. In SIGMOD. 715--730.Google Scholar
- Alexandros Koliousis, Pijika Watcharapichat, Matthias Weidlich, Luo Mai, Paolo Costa, and Peter Pietzuch. 2019. Crossbow: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers. Proc. VLDB Endow., Vol. 12, 11 (2019), 1399--1412.Google ScholarDigital Library
- Tim Kraska. 2021. Towards Instance-Optimized Data Systems. Proc. VLDB Endow., Vol. 14, 12 (2021), 3222--3232.Google ScholarDigital Library
- Tim Kraska, Mohammad Alizadeh, Alex Beutel, Ed H. Chi, Ani Kristo, Guillaume Leclerc, Samuel Madden, Hongzi Mao, and Vikram Nathan. 2019. SageDB: A Learned Database System. In CIDR.Google Scholar
- Tim Kraska, Alex Beutel, Ed H. Chi, Jeffrey Dean, and Neoklis Polyzotis. 2018. The Case for Learned Index Structures. In SIGMOD. 489--504.Google ScholarDigital Library
- Ani Kristo, Kapil Vaidya, Ugur cCetintemel, Sanchit Misra, and Tim Kraska. 2020. The Case for a Learned Sorting Algorithm. In SIGMOD. 1001--1016.Google Scholar
- Nikolina Kubiak and Simon Hadfield. 2021. TACTIC: Joint Rate-Distortion-Accuracy Optimisation for Low Bitrate Compression. CoRR, Vol. abs/2109.10658 (2021).Google Scholar
- Andreas Kunft, Asterios Katsifodimos, Sebastian Schelter, Sebastian Breß, Tilmann Rabl, and Volker Markl. 2019. An Intermediate Representation for Optimizing Machine Learning Pipelines. Proc. VLDB Endow., Vol. 12, 11 (2019), 1553--1567.Google ScholarDigital Library
- Andrew Lavin and Scott Gray. 2016. Fast Algorithms for Convolutional Neural Networks. In CVPR.Google Scholar
- Nam Le, Honglei Zhang, Francesco Cricri, Ramin Ghaznavi-Youvalari, and Esa Rahtu. 2021. Image Coding For Machines: An End-To-End Learned Approach. In 46th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1590--1594.Google Scholar
- Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. 2017. Pruning Filters for Efficient ConvNets. In ICLR.Google Scholar
- Pengfei Li, Hua Lu, Qian Zheng, Long Yang, and Gang Pan. 2020. LISA: A Learned Index Structure for Spatial Data. In SIGMOD. 2119--2133.Google Scholar
- Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Jixiang Li, Ji Liu, Ce Zhang, and Bin Cui. 2022. Hyper-Tune: Towards Efficient Hyper-Parameter Tuning at Scale. Proc. VLDB Endow., Vol. 15, 6 (2022), 1256--1265.Google ScholarDigital Library
- Yang Li, Yu Shen, Wentao Zhang, Jiawei Jiang, Bolin Ding, Yaliang Li, Jingren Zhou, Zhi Yang, Wentao Wu, Ce Zhang, and Bin Cui. 2021. VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition. Proc. VLDB Endow., Vol. 14, 11 (2021), 2167--2176.Google ScholarDigital Library
- Ji Lin, Wei-Ming Chen, Han Cai, Chuang Gan, and Song Han. 2021. Memory-efficient Patch-based Inference for Tiny Deep Learning. In NeurIPS. 2346--2358.Google Scholar
- Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, and Song Han. 2020. MCUNet: Tiny Deep Learning on IoT Devices. In NeurIPS.Google Scholar
- Ji Lin, Chuang Gan, and Song Han. 2019. Defensive Quantization: When Efficiency Meets Robustness. In ICLR.Google Scholar
- Ji Lin, Yongming Rao, Jiwen Lu, and Jie Zhou. 2017. Runtime Neural Pruning. In NeurIPS.Google Scholar
- Jinming Liu, Heming Sun, and Jiro Katto. 2021b. Learning in Compressed Domain for Faster Machine Vision Tasks. In 36th International Conference on Visual Communications and Image Processing (VCIP). 1--5.Google Scholar
- Jinming Liu, Heming Sun, and Jiro Katto. 2022b. Improving Multiple Machine Vision Tasks in the Compressed Domain. In 26th International Conference on Pattern Recognition (ICPR). 331--337.Google Scholar
- Jinming Liu, Heming Sun, and Jiro Katto. 2022c. Semantic Segmentation in Learned Compressed Domain. CoRR, Vol. abs/2209.01355 (2022).Google Scholar
- Linfeng Liu, Tong Chen, Haojie Liu, Shiliang Pu, Li Wang, and Qiu Shen. 2022a. 2C?Net: Integrate Image Compression and Classification via Deep Neural Network. Multimedia Systems (2022).Google Scholar
- Xingyu Liu, Jeff Pool, Song Han, and William J. Dally. 2018. Efficient Sparse-Winograd Convolutional Neural Networks. In ICLR.Google Scholar
- Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. 2017. Learning Efficient Convolutional Networks through Network Slimming. In ICCV. 2755--2763.Google Scholar
- Zihao Liu, Sicheng Li, Yen-kuang Chen, Tao Liu, Qi Liu, Xiaowei Xu, Yiyu Shi, and Wujie Wen. 2020. Orchestrating Medical Image Compression and Remote Segmentation Networks. In 23rd International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). 406--416.Google Scholar
- Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021a. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In ICCV. IEEE, 9992--10002.Google Scholar
- Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Ting Cheng, and Jian Sun. 2019. MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning. In ICCV. 3295--3304.Google Scholar
- Shao-Yuan Lo and Hsueh-Ming Hang. 2019. Exploring Semantic Segmentation on the DCT Representation. In 1st ACM International Conference on Multimedia in Asia (MMASIA). 1--6.Google Scholar
- Guo Lu, Xingtong Ge, Tianxiong Zhong, Jing Geng, and Qiang Hu. 2022. Preprocessing Enhanced Image Compression for Machine Vision. CoRR, Vol. abs/2206.05650 (2022).Google Scholar
- Shangyu Luo, Dimitrije Jankov, Binhang Yuan, and Chris Jermaine. 2021. Automatic Optimization of Matrix Implementations for Distributed Machine Learning and Linear Algebra. In SIGMOD. 1222--1234.Google Scholar
- Kaihao Ma, Xiao Yan, Zhenkun Cai, Yuzhen Huang, Yidi Wu, and James Cheng. 2023. FEC: Efficient Deep Recommendation Model Training with Flexible Embedding Communication. Proc. ACM Manag. Data, Vol. 1, 2 (2023), 21 pages.Google ScholarDigital Library
- Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. 2018. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. In ECCV. 122--138.Google Scholar
- Samuel Madden, Jialin Ding, Tim Kraska, Sivaprasad Sudhir, David Cohen, Timothy G. Mattson, and Nesime Tatbul. 2022. Self-Organizing Data Containers. In CIDR.Google Scholar
- Ryan Marcus, Parimarjan Negi, Hongzi Mao, Nesime Tatbul, Mohammad Alizadeh, and Tim Kraska. 2021. Bao: Making Learned Query Optimization Practical. In SIGMOD. 1275--1288.Google Scholar
- Ryan Marcus, Parimarjan Negi, Hongzi Mao, Chi Zhang, Mohammad Alizadeh, Tim Kraska, Olga Papaemmanouil, and Nesime Tatbul. 2019. Neo: A Learned Query Optimizer. Proc. VLDB Endow., Vol. 12, 11 (2019), 1705--1718.Google ScholarDigital Library
- Yixin Mei, Fan Li, Li Li, and Zhu Li. 2021. Learn A Compression for Objection Detection - VAE With A Bridge. In 36th International Conference on Visual Communications and Image Processing (VCIP). 1--5.Google Scholar
- Xupeng Miao, Xiaonan Nie, Yingxia Shao, Zhi Yang, Jiawei Jiang, Lingxiao Ma, and Bin Cui. 2021. Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce. In SIGMOD. 2262--2270.Google Scholar
- Oscar Moll, Favyen Bastani, Sam Madden, Mike Stonebraker, Vijay Gadepally, and Tim Kraska. 2022a. ExSample: Efficient Searches on Video Repositories through Adaptive Sampling. In ICDE.Google Scholar
- Oscar Moll, Manuel Favela, Samuel Madden, and Vijay Gadepally. 2022b. SeeSaw: Interactive Ad-hoc Search Over Image Databases. CoRR, Vol. abs/2208.06497 (2022).Google Scholar
- Paul Mooney. 2018. Blood Cell Images. https://www.kaggle.com/datasets/paultimothymooney/blood-cells Accessed on May 16, 2023.Google Scholar
- Supun Nakandala and Arun Kumar. 2022. Nautilus: An Optimized System for Deep Transfer Learning over Evolving Training Datasets. In SIGMOD. 506--520.Google Scholar
- Supun Nakandala, Yuhao Zhang, and Arun Kumar. 2020. Cerebro: A Data System for Optimized Deep Learning Model Selection. Proc. VLDB Endow., Vol. 13, 12 (2020), 2159--2173.Google ScholarDigital Library
- Vikram Nathan, Jialin Ding, Mohammad Alizadeh, and Tim Kraska. 2020. Learning Multi-Dimensional Indexes. In SIGMOD. 985--1000.Google Scholar
- Xiaonan Nie, Xupeng Miao, Zilong Wang, Zichao Yang, Jilong Xue, Lingxiao Ma, Gang Cao, and Bin Cui. 2023. FlexMoE: Scaling Large-Scale Sparse Pre-Trained Model Training via Dynamic Device Placement. Proc. ACM Manag. Data, Vol. 1, 1 (2023), 19 pages.Google ScholarDigital Library
- Kwanghyun Park, Karla Saur, Dalitso Banda, Rathijit Sen, Matteo Interlandi, and Konstantinos Karanasos. 2022. End-to-End Optimization of Machine Learning Prediction Queries. In SIGMOD. 587--601.Google Scholar
- Neel Patwa, Nilesh Ahuja, Srinivasa Somayazulu, Omesh Tickoo, Srenivas Varadarajan, and Shashidhar Koolagudi. 2020. Semantic-Preserving Image Compression. In 27th IEEE International Conference on Image Processing (ICIP). 1281--1285.Google Scholar
- Jingshu Peng, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, and Jiannong Cao. 2022. Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks. Proc. VLDB Endow., Vol. 15, 9 (2022), 1937--1950.Google ScholarDigital Library
- Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient Neural Architecture Search via Parameters Sharing. In ICML. 4095--4104.Google Scholar
- Arnab Phani, Lukas Erlbacher, and Matthias Boehm. 2022. UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads. Proc. VLDB Endow., Vol. 15, 11 (2022), 2929--2938.Google ScholarDigital Library
- Arnab Phani, Benjamin Rath, and Matthias Boehm. 2021. LIMA: Fine-Grained Lineage Tracing and Reuse in Machine Learning Systems. In SIGMOD. 1426--1439.Google Scholar
- Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. 2016. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. In ECCV. 525--542.Google Scholar
- Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. 2017. Snorkel: Rapid Training Data Creation with Weak Supervision. Proc. VLDB Endow., Vol. 11, 3 (2017), 269--282.Google ScholarDigital Library
- Luis Remis and Chaunté W. Lacewell. 2021. Using VDMS to Index and Search 100M Images. Proceedings of the VLDB Endowment, Vol. 14, 12 (2021), 3240--3252.Google ScholarDigital Library
- Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In NIPS. 91--99.Google Scholar
- Alexander Renz-Wieland, Rainer Gemulla, Zoi Kaoudi, and Volker Markl. 2022. NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access. In SIGMOD. 481--495.Google Scholar
- Alexander Renz-Wieland, Rainer Gemulla, Steffen Zeuch, and Volker Markl. 2020. Dynamic Parameter Allocation in Parameter Servers. Proc. VLDB Endow., Vol. 13, 12 (2020), 1877--1890.Google ScholarDigital Library
- Ties Robroek, Aaron Duane, Ehsan Yousefzadeh-Asl-Miandoab, and Pinar Tozun. 2023. Data Management and Visualization for Benchmarking Deep Learning Training Systems. In DEEM Workshop.Google Scholar
- Samuel Felipe dos Santos, Nicu Sebe, and Jurandy Almeida. 2020. The Good, The Bad, and The Ugly: Neural Networks Straight From JPEG. In 27th IEEE International Conference on Image Processing (ICIP). 1896--1900.Google ScholarCross Ref
- Sebastian Schelter, Stefan Grafberger, and Ted Dunning. 2021. HedgeCut: Maintaining Randomised Trees for Low-Latency Machine Unlearning. In SIGMOD. 1545--1557.Google Scholar
- Sebastian Schelter, Tammo Rukat, and Felix Biessmann. 2020. Learning to Validate the Predictions of Black Box Classifiers on Unseen Data. In SIGMOD. 1289--1299.Google Scholar
- Saurabh Singh, Sami Abu-El-Haija, Nick Johnston, Johannes Ballé, Abhinav Shrivastava, and George Toderici. 2020. End-to-End Learning of Compressible Features. In 27th IEEE International Conference on Image Processing (ICIP). 3349--3353.Google Scholar
- Utku Sirin and Stratos Idreos. 2024. The Image Calculator Technical Report. (2024).Google Scholar
- Samuel L. Smith, Andrew Brock, Leonard Berrada, and Soham De. 2023. ConvNets Match Vision Transformers at Scale. CoRR, Vol. abs/2310.16764 (2023).Google Scholar
- Simeng Sun, Tianyu He, and Zhibo Chen. 2021. Semantic Structured Image Coding Framework for Multiple Intelligent Applications. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, 9 (2021), 3631--3642.Google ScholarCross Ref
- Peter Symes. 1998. Video Compression: Fundamental Compression Techniques and an Overview of the JPEG and MPEG Compression Systems. McGraw-Hill.Google Scholar
- Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le. 2019. MnasNet: Platform-Aware Neural Architecture Search for Mobile. In CVPR. 2820--2828.Google Scholar
- Mingxing Tan and Quoc V. Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In ICML. 6105--6114.Google Scholar
- Róbert Torfason, Fabian Mentzer, Eiríkur Ágústsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. 2018. Towards Image Understanding from Deep Compression Without Decoding. In ICLR. 1--17.Google Scholar
- Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan C. Bovik, and Yinxiao Li. 2022. MaxViT: Multi-axis Vision Transformer. In ECCV. 459--479.Google Scholar
- Paroma Varma and Christopher Ré. 2018. Snuba: Automating Weak Supervision to Label Training Data. Proc. VLDB Endow., Vol. 12, 3 (2018), 223--236.Google ScholarDigital Library
- Xinchen Wan, Kaiqiang Xu, Xudong Liao, Yilun Jin, Kai Chen, and Xin Jin. 2023. Scalable and Efficient Full-Graph GNN Training for Large Graphs. Proc. ACM Manag. Data, Vol. 1, 2 (2023), 23 pages.Google ScholarDigital Library
- Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han. 2019b. HAQ: Hardware-Aware Automated Quantization With Mixed Precision. In CVPR. 8612--8620.Google Scholar
- Shurun Wang, Zhao Wang, Shiqi Wang, and Yan Ye. 2021. End-to-End Compression Towards Machine Vision: Network Architecture Design and Optimization. IEEE Open Journal of Circuits and Systems, Vol. 2, 1 (2021), 675--685.Google ScholarCross Ref
- Tianzhe Wang, Kuan Wang, Han Cai, Ji Lin, Zhijian Liu, Hanrui Wang, Yujun Lin, and Song Han. 2020. APQ: Joint Search for Network Architecture, Pruning and Quantization Policy. In CVPR. 2075--2084.Google Scholar
- Zeke Wang, Kaan Kara, Hantian Zhang, Gustavo Alonso, Onur Mutlu, and Ce Zhang. 2019a. Accelerating Generalized Linear Models with MLWeaving: A One-Size-Fits-All System for Any-Precision Learning. Proc. VLDB Endow., Vol. 12, 7 (2019), 807--821.Google ScholarDigital Library
- Zixi Wang, Fan Li, Jing Xu, and Pamela C. Cosman. 2022a. Human--Machine Interaction-Oriented Image Coding for Resource-Constrained Visual Monitoring in IoT. IEEE Internet of Things Journal, Vol. 9, 17 (2022), 16181--16195.Google ScholarCross Ref
- Zhenzhen Wang, Minghai Qin, and Yen-Kuang Chen. 2022b. Learning From the CNN-based Compressed Domain. In 22nd IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 4000--4008.Google Scholar
- Maurice Weber, Cedric Renggli, Helmut Grabner, and Ce Zhang. 2020. Observer Dependent Lossy Image Compression. In 8th DAGM German Conference on Pattern Recognition (GCPR). 130--144.Google Scholar
- Nathaniel Weir, Prasetya Utama, Alex Galakatos, Andrew Crotty, Amir Ilkhechi, Shekar Ramaswamy, Rohin Bhushan, Nadja Geisler, Benjamin H"attasch, Steffen Eger, Ugur Cetintemel, and Carsten Binnig. 2020. DBPal: A Fully Pluggable NL2SQL Training Pipeline. In SIGMOD. 2347--2361.Google Scholar
- Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. 2019. FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search. In CVPR. 10734--10742.Google Scholar
- Yuncheng Wu, Tien Tuan Anh Dinh, Guoyu Hu, Meihui Zhang, Yeow Meng Chee, and Beng Chin Ooi. 2022. Serverless Data Science - Are We There Yet? A Case Study of Model Serving. In SIGMOD. 1866--1875.Google Scholar
- Jiuhong Xiao, Lavisha Aggarwal, Prithviraj Banerjee, Manoj Aggarwal, and Gerard Medioni. 2022. Identity Preserving Loss for Learned Image Compression. In CVPR Workshop on New Trends in Image Restoration and Enhancement and Challenges. 517--526.Google Scholar
- Doris Xin, Stephen Macke, Litian Ma, Jialin Liu, Shuchen Song, and Aditya Parameswaran. 2018. HELIX: Holistic Optimization for Accelerating Iterative Machine Learning. Proc. VLDB Endow., Vol. 12, 4 (2018), 446--460.Google ScholarDigital Library
- Kai Xu. 2021. Learning in Compressed Domains. Ph.,D. Dissertation. Arizona State University.Google Scholar
- Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren. 2020. Learning in the Frequency Domain. In CVPR. 1740--1749.Google Scholar
- Zhiqiang Xu, Dong Li, Weijie Zhao, Xing Shen, Tianbo Huang, Xiaoyun Li, and Ping Li. 2021. Agile and Accurate CTR Prediction Model Training for Massive-Scale Online Advertising Systems. In SIGMOD. 2404--2409.Google Scholar
- Shuai Yang, Yueyu Hu, Wenhan Yang, Ling-Yu Duan, and Jiaying Liu. 2021. Towards Coding for Human and Machine Vision: Scalable Face Image Coding. IEEE Transactions on Multimedia, Vol. 23, 1 (2021), 2957--2971.Google ScholarCross Ref
- Gyeong-In Yu, Saeed Amizadeh, Sehoon Kim, Artidoro Pagnoni, Ce Zhang, Byung-Gon Chun, Markus Weimer, and Matteo Interlandi. 2021. WindTunnel: Towards Differentiable ML Pipelines beyond a Single Model. Proc. VLDB Endow., Vol. 15, 1 (2021), 11--20.Google ScholarDigital Library
- Binhang Yuan, Cameron R. Wolfe, Chen Dun, Yuxin Tang, Anastasios Kyrillidis, and Chris Jermaine. 2022. Distributed Learning of Fully Connected Neural Networks Using Independent Subnet Training. Proc. VLDB Endow., Vol. 15, 8 (2022), 1581--1590.Google ScholarDigital Library
- Xin Zhang, Yanyan Shen, Yingxia Shao, and Lei Chen. 2023. DUCATI: A Dual-Cache Training System for Graph Neural Networks on Giant Graphs with the GPU. Proc. ACM Manag. Data, Vol. 1, 2 (2023).Google ScholarDigital Library
- Yuhao Zhang and Arun Kumar. 2019. Panorama: A Data System for Unbounded Vocabulary Querying over Video. Proc. VLDB Endow., Vol. 13, 4 (2019), 477--491.Google ScholarDigital Library
- Yuhao Zhang, Frank McQuillan, Nandish Jayaram, Nikhil Kak, Ekta Khanna, Orhan Kislal, Domino Valdano, and Arun Kumar. 2021. Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches. Proc. VLDB Endow., Vol. 14, 10 (2021), 1769--1782.Google ScholarDigital Library
- Chenguang Zheng, Hongzhi Chen, Yuxuan Cheng, Zhezheng Song, Yifan Wu, Changji Li, James Cheng, Hao Yang, and Shuai Zhang. 2022. ByteGNN: Efficient Graph Neural Network Training at Large Scale. Proc. VLDB Endow., Vol. 15, 6 (2022), 1228--1242.Google ScholarDigital Library
- Hongkuan Zhou, Da Zheng, Israt Nisa, Vasileios Ioannidis, Xiang Song, and George Karypis. 2022. TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs. Proc. VLDB Endow., Vol. 15, 8 (2022), 1572--1580.Google ScholarDigital Library
- Shuchang Zhou, Zekun Ni, Xinyu Zhou, He Wen, Yuxin Wu, and Yuheng Zou. 2016. DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients. CoRR, Vol. abs/1606.06160 (2016).Google Scholar
- Chenzhuo Zhu, Song Han, Huizi Mao, and William J. Dally. 2017. Trained Ternary Quantization. In ICLR.Google Scholar
Index Terms
- The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format
Recommendations
A document image model and estimation algorithm for optimized JPEG decompression
The JPEG standard is one of the most prevalent image compression schemes in use today. While JPEG was designed for use with natural images, it is also widely used for the encoding of raster documents. Unfortunately, JPEG's characteristic blocking and ...
DCT Sign-Based Similarity Measure for JPEG Image Retrieval
We propose a method to retrieve similar and duplicate images from a JPEG (Joint Photographic Image Group) image database. Similarity level is decided based on the DCT (Discrete Cosine Transform) coefficients signs. The method is simple and fast because ...
An Approach for Color Image Compression of JPEG and PNG Images Using DCT and DWT
CICN '14: Proceedings of the 2014 International Conference on Computational Intelligence and Communication NetworksNow a days image compression has become is an indispensible part of digitized image storage and transmission. Compression of an image is necessary before storing and transmitting it due to its limitation of storage and bandwidth capacity. Wavelet ...
Comments