"Impact Factors" in System Research

Definition

Impact Factor is commonly used to measure the importance of a journal by calculating the number of times that articles in the journal are cited within the last few years. However, this metric is too narrow to measure the real impact of system research in computer science.

In my opinion, the impact of system research should have more comprehensive measurements to reflect its value to academia and industry. Specifically, we have the following “Impact Factors” for system research.

Citations: This metric is usually measured in the number of citations from GoogleScholar.
Relevance to Industry and Open-Source Community: Whether the system research has inspired patents and industry products and cultivated the growth of open-source community.
System Repeatability and Academic Impacts: A good system prototype should be repeatable in other publications’ experiments, and have inspirations to future system publications. We consider this metric as “Influential Citations” in system research. For example, IEEE Transactions on Parallel and Distributed Systems (TPDS) has recently created an Reproducibility Editorial Board to coordinate the review of the code.
Educational Adoptions: How the system research has impacted education/academia in terms of text books and reading lists in courses/seminars.
Media Coverage: How the system research has reached much broader audience in social media.

Overview of Our Research

Here are some highlights of our past projects with high impact factors:

Citations: A number of our papers are highly cited in the field. One highlight is that our Mars paper in ACM PACT 2008 is ranked No. 2 among all the papers published in ACM PACT (since 1993), according to ACM Digital Library.
Relevance to Industry and Open-Source Community: Our research has attracted interests from many companies such as Microsoft, NVIDIA, Intel, Xilinx, Huawei, Grab, Bytedance (in the forms of gift grants, collaboration projects, hardware donations and consultancy). ThunderSVM and ThunderGBM have got over 1,100 stars and 500 stars respectively since the projects are released in less than two years. The number of unique visitors monthly are 800 and 300, respectively.
System Repeatability and Academic Impacts: Our system prototypes (such as Medusa, OmniDB, Mars and FD-tree) are often requested and used by other researchers.
Educational Adoptions: We have observed the wide spread of our research into education/academia in terms of references in many books and having been widely used in courses/seminars.
Media Coverage: ThunderSVM has been highlighted in headline of popular open-source websites including Hacker News and Packt DataHub. ThunderGBM has been highlighted in Reddit, jiqizhixin with over 5000 repostings.