IVE's big data apps create biz values from govt open data

Hong Kong IT Job Advertisement Data Mining ReportHong Kong IVE cloud majors show big data analytics in action by creating business values out of government open data.

What are the most sought-after IT skills in Hong Kong today?

What is the most popular programming language among employers?

What additional programming languages does a JAVA-proficient job candidate ought to know?

If someone is proficient in C++, which district in Hong Kong will he likely find the most job matches? 

More importantly, can you answer the above questions accurately without surveying all the job ads of the week? 

Students doing a cloud major at the Hong Kong Institute of Vocational Education (IVE) can. Indeed, any employer or IT job candidate can answer similar queries with the help of, a cloud data applications platform created by an award-winning student team at IVE. 

Cyrus Wong, R&D coordinator, department of multimedia and internet technology, Hong Kong IVE

"Our work [on Data-HK] is not a student assignment or project, but real production work that benefits the society!" 

-- Cyrus Wong, R&D coordinator, department of multimedia and internet technology, Hong Kong IVE

Data-HK is a big data analytics project that comprises five cloud-based applications, which provide open access to live public data for research and business purposes. The various data feeds for the big data analytics applications include: 1) Data.One -- public data sets released by the Hong Kong Government; 2) internet keyword analysis from web content; and 3) scientific data from the business sector. To support its huge computation needs, the IVE team uses Amazon Web Services' public cloud services (AWS) to build unlimited capacity to share public data sets. 

The mastermind behind Data-HK is Cyrus Wong, research and development coordinator, department of multimedia and internet technology at Hong Kong IVE (Lee Wai Lee), plus eight students who are currently taking a two-year full-time program called the Higher Diploma in Cloud and Data Centre Administration. 

Big data analytics in action 

The primary source of data for Data-HK, Data.One, was launched by the Office of Government Chief Information Officer (OGCIO) in March 2011. Data.One provides geo-referenced public facilities data and real-time traffic data for free download and value-added reuse by the public. 

Since Data.One provides only real-time data in disparate file formats (.xml, csv, json, etc.), this gave rise to Data.Two, a "user friendly version" of Data.One produced by the Data-HK team. Unlike Data.One, Data.Two archives all Data.One datasets and converts them from inconsistent data formats to unified Restful API for easier data retrieval. 

On top of Data.Two, the Data-HK team has built five different big data analytics applications. The first three projects below uses public data sets from Data.One, while the last two projects use other publicly available data:

- LicenCheck (an Android and web app that provides a map view of Hong Kong with markers locating all licensed restaurants);

- Missing HK (an Android app that lists all wanted persons in Hong Kong);

- HK Traffic Live (a web app that renders real-time traffic and weather information on a Hong Kong map);

- IT Jobs Analysis (a web app that uses data science to investigate and extract the keywords from 192,000 IT job ads to help make informed decisions about one's IT career development); and

- DSE English Learning (An Android App that text analytics and word database to find the most commonly used words appeared in past HKDSE English Exam) 

Value creation takes priority 

"These big data analytics applications add value to the existing government datasets," Wong said. "Take LicenCheck for example, while the government provides the addresses of licensed restaurants, Data-HK converts the addresses into geo location searchable on a map." According to Wong, inspectors at the Food and Environmental Hygiene Department can inspect restaurant licenses efficiently by following the suggested routes on the map. 

"Data-HK aims to investigate the application potential of the PSI (Public Sector Information) deeper and wider as we realize the huge potential values of PSI. Through these applications, it will also encourage the government departments to disclose more their data to the public," Wong said. 

"Based on these open data, more and more applications will be developed and more people in the society will be benefited. We will keep a close cooperation with the government departments, corporations, and public media to develop Hong Kong's knowledge-based economy." 

Wong reiterated: "Our work [on Data-HK] is not a student assignment or project, but real production work that benefits the society!"