Research Projects

End host-based tools, such as speed tests, are widely used by end-users, network operators, and researchers to assess and diagnose network performance. Moreover, data aggregated across multiple measurements can help policymakers and regulators to identify gaps in Internet availability and regulate ISPs. For these tools to be really useful, they should satisfy the following properties: 1). accurate, report correct metrics; 2). efficient, minimize measurement overhead to prevent disruption of background applications and enable longitudinal data collection, 3) contextualized, provide actionable insights, such as identify the location of bottleneck link being measured, and 4) support longitudinal measurements that is critical for characterizing dynamic networks. While significant efforts have been made to improve the accuracy of network diagnostic tools, insufficient attention has been given to the other three design goals, thus, constraining the overall utility of these tools. For example, the prevalent speed test tools incur a significant network overhead, especially in high-speed networks, posing challenges for longitudinal measurements in networks with data caps. Goal: The goal of this proposal is to build a toolbox of data-driven network diagnostics techniques that are not just accurate but also efficient, contextualized, and capable of providing longitudinal measurements. Proposed Approach: Our proposed methods (detailed in the technical document) are based on two key insights. First, specially crafted lightweight probes can provide valuable information about the underlying network context. Specifically, these probes can complement traditional heavyweight measurements by offering additional context. However, while lightweight probes are informative, they are insufficient on their own due to the complexity of modeling their relationship with the hidden network context, especially given the wide distribution of network conditions (e.g., background traffic, network configuration) in today’s Internet. To address this, we propose developing novel learning problems that leverage state-of-the-art machine learning techniques to effectively infer hidden network contexts from lightweight probes. A significant challenge also lies in training and validating these ML models, which require high-quality data. To meet this need, we will build data collection systems capable of gathering data from diverse resources including residential and campus networks. Impact: This proposal will significantly enhance the utility of speed test tools, one of the most widely used network diagnostic tools. There is great potential for technology transfer, either by integrating our methods with existing open-source tools like NDT or creating a new, indigenous network diagnostic tool. Additionally, our work directly addresses policy issues related to digital inclusion by enabling communities and policymakers to collect longitudinal, actionable data on network performance.