The advances in high-throughput sensing techniques have shifted the paradigm of many research fields with the unprecedented capability in data collection. With the availability of diverse types of “big” data as well as powerful computation, data-driven research with recent successes of “deep” learning in many artificial intelligence applications, has been believed by many to make new breakthroughs in advancing science and maximizing health and economy outcomes. On the other hand, when studying complex systems, we often face challenges in learning under data scarcity, due to either prohibitive cost or inherent difficulty in obtaining required training samples with respect to the system complexity and uncertainty. In fact, the main concern of data-driven research when studying complex systems, either man-made or nature systems, has been the reproducibility of the scientific findings. In addition to many practical big-data analytic challenges, there still lack rigorous theoretical guidelines in “big” data and “deep” learning research when studying complex systems. With a brief overview of the recent and ongoing projects with my students, I would like to present our several machine learning efforts under data-poor environments and share my current understanding of theoretical, modeling, and analytic challenges in learning complex systems.