Shanu Kumar
ABSTRACT
Citizen engagement and technology usage are two emerging trends driven by smart city initiatives. Typically, citizens report issues, such as broken roads, garbage dumps, etc. through web portals and mobile apps, in order for the government authorities to take appropriate actions. Several mediums - text, image, audio, video - are used to report these issues. Through a user study with 13 citizens and 3 authorities, we found that image is the most preferred medium to report civic issues. However, analyzing civic issue related images is challenging for the authorities as it requires manual effort. In this work, given an image, we propose to generate a Civic Issue Graph consisting of a set of objects and the semantic relations between them, which are representative of the underlying civic issue. We also release two multi-modal (text and images) datasets, that can help in further analysis of civic issues from images. We present an approach for adversarial adaptation of existing scene graph models that enables the use of scene graphs for new applications in the absence of any labelled training data. We conduct several experiments to analyze the efficacy of our approach, and using human evaluation, we establish the appropriateness of our model at representing different civic issues.
- 2016. FixMyStreet. (2016). https://www.fixmystreet.com/Google Scholar
- 2017. Mayor's Management Report. https://www1.nyc.gov/assets/operations/downloads/pdf/mmr2017/2017_mmr.pdf. (2017).Google Scholar
- Deborah Agostino. 2013. Using social media to engage citizens: A study of Italian municipalities. Public Relations Review39, 3 (2013), 232-234.Google Scholar
- Shubham Atreja, Pooja Aggarwal, Prateeti Mohapatra, Amol Dumrewal, Anwesh Basu, and Gargi B Dasgupta. 2018. Citicafe: An Interactive Interface for Citizen Engagement. In 23rd International Conference on Intelligent User Interfaces. ACM, 617-628. Google ScholarDigital Library
- Tseng-Hung Chen, Yuan-Hong Liao, Ching-Yao Chuang, Wan Ting Hsu, Jianlong Fu, and Min Sun. 2017. Show, Adapt and Tell: Adversarial Training of Cross-Domain Image Captioner.. In ICCV. 521-530.Google Scholar
- Tseng-Hung Chen, Yuan-Hong Liao, Ching-Yao Chuang, Wan Ting Hsu, Jianlong Fu, and Min Sun. 2017. Show, Adapt and Tell: Adversarial Training of Cross-Domain Image Captioner.. In ICCV. 521-530.Google Scholar
- Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. 2018. Domain Adaptive Faster R-CNN for Object Detection in the Wild. In Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Annalisa Cocchia. 2014. Smart and digital city: A systematic literature review. In Smart city. Springer, 13-43.Google Scholar
- Peter Dahlgren. 2011. Parameters of online participation: Conceptualising civic contingencies. Communication management quarterly21, 4 (2011), 87-110.Google Scholar
- Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised Domain Adaptation by Backpropagation. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37(ICML'15). JMLR.org, 1180-1189. http://dl.acm.org/citation.cfm?id=3045118.3045244 Google ScholarDigital Library
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 2672-2680. http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf Google ScholarDigital Library
- Lushan Han, Abhay L Kashyap, Tim Finin, James Mayfield, and Jonathan Weese. 2013. UMBC_EBIQUITY-CORE: Semantic textual similarity systems. In Second Joint Conference on Lexical and Computational Semantics (* SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity, Vol. 1. 44-52.Google Scholar
- Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. 2015. Image retrieval using scene graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3668-3678.Google ScholarCross Ref
- Maria Karakiza. 2015. The impact of social media in the public sector. Procedia-Social and Behavioral Sciences175 (2015), 384-392.Google Scholar
- Matthew Klawonn and Eric Heim. 2018. Generating Triples with Adversarial Networks for Scene Graph Construction. arXiv preprint arXiv:1802.02598(2018).Google Scholar
- Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, and others. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of Computer Vision123, 1 (2017), 32-73. Google ScholarDigital Library
- Shanu Kumar, Shubham Atreja, Anjali Singh, and Mohit Jain. 2019. Adversarial Adaptation of Scene Graph Models for Understanding Civic Issues. arXiv preprint arXiv:1901.10124(2019). Google ScholarDigital Library
- Kongming Liang, Yuhong Guo, Hong Chang, and Xilin Chen. 2018. Visual Relationship Detection with Deep Structural Ranking. (2018).Google Scholar
- Hiroya Maeda, Yoshihide Sekimoto, Toshikazu Seto, Takehiro Kashiyama, and Hiroshi Omata. 2018. Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone. arXiv preprint arXiv:1801.09454(2018).Google Scholar
- Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations. 55-60.Google ScholarCross Ref
- Graeme Mearns, Rebecca Simmonds, Ranald Richardson, Mark Turner, Paul Watson, and Paolo Missier. 2014. Tweet my street: a cross-disciplinary collaboration for the analysis of local twitter data. Future Internet6, 2 (2014), 378-396.Google Scholar
- Ines Mergel. 2012. Distributed democracy: Seeclickfix. com for crowdsourced issue reporting. (2012).Google Scholar
- Gaurav Mittal, Kaushal B Yagnik, Mohit Garg, and Narayanan C Krishnan. 2016. Spotgarbage: smartphone app to detect garbage using deep learning. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 940-945. Google ScholarDigital Library
- Nitish Mittal, Swati Agarwal, and Ashish Sureka. 2016. Got a Complaint?-Keep Calm and Tweet It!. In International Conference on Advanced Data Mining and Applications. Springer, 619-635.Google ScholarDigital Library
- Taewoo Nam and Theresa A Pardo. 2011. Conceptualizing smart city with dimensions of technology, people, and institutions. In Proceedings of the 12th annual international digital government research conference: digital government innovation in challenging times. ACM, 282-291. Google ScholarDigital Library
- Paolo Neirotti, Alberto De Marco, Anna Corinna Cagliano, Giulio Mangano, and Francesco Scorrano. 2014. Current trends in Smart City initiatives: Some stylised facts. Cities38(2014), 25-36.Google Scholar
- Maxime Oquab, Leon Bottou, Ivan Laptev, and Josef Sivic. 2014. Learning and transferring mid-level image representations using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1717-1724. Google ScholarDigital Library
- Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. 2018. Multi-Adversarial Domain Adaptation. In AAAI.Google Scholar
- Sebastian Schuster, Ranjay Krishna, Angel Chang, Li Fei-Fei, and Christopher D Manning. 2015. Generating semantically precise scene graphs from textual descriptions for improved image retrieval. In Proceedings of the fourth workshop on vision and language. 70-80.Google ScholarCross Ref
- Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Computer Vision and Pattern Recognition (CVPR), Vol. 1. 4.Google Scholar
- Danfei Xu, Yuke Zhu, Christopher B Choy, and Li Fei-Fei. 2017. Scene graph generation by iterative message passing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2.Google ScholarCross Ref
- Rowan Zellers, Mark Yatskar, Sam Thomson, and Yejin Choi. 2018. Neural Motifs: Scene Graph Parsing with Global Context. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5831-5840.Google ScholarCross Ref
- Bohan Zhuang, Qi Wu, Chunhua Shen, Ian D Reid, and Anton van den Hengel. 2018. HCVRD: A Benchmark for Large-Scale Human-Centered Visual Relationship Detection.. In AAAI.Google Scholar
Recommendations
The democratic potential of civic applications
Recently, digital democratic applications have increased in presence and scope. This study clarifies how civic applications – bottom-up technologies that use open data to solve governance and policy challenges – can contribute to democratic ...
Civic innovation as a response to social problems: the case of Civic and public challenges in Mexico
TEEM '16: Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing MulticulturalityIn this paper, the Civic Challenges and Public Challenges initiatives developed in Mexico, as an innovation experience on the implementation of technological strategies for the solution of social problems, are described.
We will reflect on the concept of ...
HCI, Civic Engagement & Trust
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsThere is a widespread belief that pervasive technologies will encourage and facilitate partnerships between citizens and civic authorities, enabling individuals to play a greater role in civic planning, service delivery and infrastructure management. ...
Comments