Daniel McKenzie
Applied Math and ML.
Positions
2022--present Colorado School of Mines. Assistant Professor
2019-2022 University of California, Los Angeles. Assistant Adjunct Professor
Research interests
Optimization and signal processing applied to machine learning and data science. More specifically: zeroth-order optimization and applications, Learn-to-optimize, and data-driven metrics for unsupervised learning.
Education
2013-2019 University of Georgia, USA. PhD. Advisor: Ming-Jun Lai.
03/2013-06/2013 University of Bayreuth, Germany. Short term visitor.
2011-2013
University of Cape Town, South Africa. M.Sc (with distinction).
2007-2010
University of Cape Town, South Africa. B.Sc (with distinction)
Awards and Recognitions
2024 Featured in Mines research magazine.
2021 Liggett Instructor Award.
2019 William Armor Wills Memorial Scholarship.
2017 University Outstanding Teaching Assistant Award.
2014-2016 NRF Doctoral Scholarship for Study Abroad.
2013 DAAD Short Term Research Exchange (to University of Bayreuth).
2012 DAAD-NRF Joint Masters Bursary.
2010 UCT Council Merit Scholarship.
2010 NRF Honours Bursary.
2009 Jakob Burlak Memorial Trust Scholarship.
2007,2008,2009 UCT Science Faculty Scholarship.
Publications
A list is also available online. *= undergraduate coauthor.
Journals
2024 Differentiating through integer linear programs with quadratic regularization and Davis-Yin splitting Daniel McKenzie, Howard Heaton, Samy Wu Fung
2024 Normalizing Basis Functions: Approximate Stationary Models for Large Spatial Data. STAT. Antony Sikorski, Daniel McKenzie, Doug Nychka.
2024 Three-Operator Splitting for Learning to Predict Equilibria in Convex Games. SIMODS. Daniel McKenzie and Howard Heaton, Qiuwei Li, Samy Wu Fung, Stanley Osher and Wotao Yin.
2024 Fermat Distances: Metric Approximation, Spectral Convergence, and Clustering Algorithms. JMLR. Nicolás García Trillos, Anna Little, Daniel McKenzie, and James Murphy (alphabetical).
2023 From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrization Information and Inference. Qiuwei Li, Daniel McKenzie, and Wotao Yin (alphabetical).
2022 A one-bit, comparison-based gradient estimator. ACHA. HanQin Cai, Daniel McKenzie, Wotao Yin and Zhenliang Zhang (alphabetical).
2021 Balancing geometry and density: Path distances on high-dimensional data. SIMODS. Anna Little, Daniel McKenzie, and James M. Murphy (alphabetical).
2021 Zeroth-order regularized optimization (ZORO): Approximately sparse gradients and adaptive sampling. SIOPT. HanQin Cai, Daniel McKenzie, Wotao Yin and Zhenliang Zhang (alphabetical).
2020 Compressive sensing for cut improvement and local clustering. SIMODS. Ming-Jun Lai and Daniel McKenzie (alphabetical).
2019 Power weighted shortest paths for clustering Euclidean data. Foundations of Data Science. Daniel McKenzie and Steve Damelin
Conferences
2022 JFB: Jacobian-Free Backpropagation for Implicit Networks. AAAI 2022 (15% acceptance rate). Samy Wu Fung, Howard Heaton, Qiuwei Li, Daniel McKenzie, Stanley Osher and Wotao Yin.
2021 A zeroth-order, block coordinate descent algorithm for huge-scale black-box optimization. ICML 2021 (22% acceptance rate). HanQin Cai, Yuchen Lou*, Daniel McKenzie, and Wotao Yin (alphabetical).
2020 Who killed Lilly Kane? A case study in applying knowledge graphs to crime fiction. IEEE Big Data GTA3 Workshop. With Mariam Alaverdian*, William Gilroy*, Veronica Kirgios*, Xia Li, Carolina Matuk*, Daniel McKenzie, Tachin Ruangkriengsin*, P. Jeffrey Brantingham and Andrea Bertozzi (alphabetical).
Submitted
2025 Fully Adaptive Zeroth-Order Method for Minimizing Functions with Compressible Gradients Geovani Nunes Grapiglia, Daniel McKenzie
2024 On logical extrapolation for mazes with recurrent and implicit networks Brandon Knutson, Amandin Rabeendran, Michael Ivanitskiy, Jordan Pettyjohn, Cecilia Deniz-Behn, Samy Wu Fung, Daniel McKenzie
2023 It begins with a boundary: A geometric view on probabilistically robust learning Leon Bungert, Nicolás García Trillos, Matt Jacobs, Daniel McKenzie, Đorđe Nikolić, Qingsong Wang.
2023Curvature-Aware Derivative-Free Optimization Bumsu Kim, Daniel McKenzie, HanQin Cai and Wotao Yin.
Misc.
2023 Adapting Zeroth Order Algorithms for Comparison-Based Optimization Isha Slavin* and Daniel McKenzie SIURO.
2022 Learning to Optimize SIAM News. Samy Wu Fung, Daniel McKenzie, Wotao Yin.
Talks
All talks invited unless otherwise stated.
10/2024 SIAM MDS. Atlanta, USA.
07/2024 IMSI Workshop on Computational Imaging. Chicago, USA.
06/2024 LOL24 Workshop. Luminy, France
03/2024 Becker Group Seminar, CU. Boulder, USA.
01/2024 CSU IDA Seminar. Fort Collins, USA.
09/2023 U. Arizona Early Career Colloquium. Online
06/2023 SIAM OP: Advances in optimization with Machine Learning. Seattle, USA.
02/2023 NIST AI COI Seminar. Boulder, USA.
02/2023 Emory CODES Seminar. Atlanta, USA.
02/2023 UGA Applied Math Seminar. Athens, USA.
10/2022 George Mason University CMAI Colloquium. Fairfax, VA.
09/2022 SIAM MDS: Manifold Learning and Dimensionality Reduction. San Diego, USA.
03/2022 SIAM UQ22: Deep Learning for Optimization. Atlanta, USA.
03/2022 Math Machine Learning Seminar. Max Planck Institute, Germany.
10/2021 INFORMS2021: Recent Advances in Derivative-free Optimization. Anaheim, USA.
08/2021 Mathematics of Machine Learning Conference. Bielefeld University, Germany. (contributed)
07/2021 SIAM OP21: Optimization, Data Science and their Applications. Online.
06/2021 Optimal Transport and Mean Field Games Seminar. Online.
11/2020 INFORMS2020: Session on Recent Progress in Blackbox Optimization. Online.
04/2020 Tufts Math of Data Science Lecture Series. Online.
02/2020 UGA Applied Math Seminar. Athens, USA.
09/2019 SIAM South-Eastern Sectional. Knoxville, USA.
05/2018 International Conference on Computational Harmonic Analysis (ICCHA7). Nashville, USA. (contributed)
10/2018 AMS Central Sectional. Ann Arbor, USA.
Teaching
2024 Math 500: Linear Spaces (graduate level), Mines
2024 Math 551: Computational Linear Algebra (graduate level), Mines
2023 Math 332: Linear Algebra, Mines.
2023 Math 599: Analysis (independent study).
2023 Math 551: Computational Linear Algebra (graduate level), Mines
2022 Math 332: Linear Algebra, Mines.
2021--2022 Math 151BH: Honors Applied Numerical Methods II, UCLA. (Taught twice. I also co-developed this course).
2021--2022 Math 151AH: Honors Applied Numerical Methods I, UCLA. (Taught twice. I also co-developed this course).
2020--2022 Math 118: Mathematical Methods of Data Theory, UCLA. (Taught four times. I also co-developed this course).
2020 Math 170S: Statistics, UCLA.
2019 Math 151A: Applied Numerical Methods I, UCLA.
2019 Math 32A: Calculus III, UCLA.
2015--2019 Math2250: Calculus I, UGA. (taught three times).
2014--2018 Math1113: Precalculus, UGA. (taught six times).
Service
Committees
2024--present Graduate Committee. Mines.
2024 Linear Algebra Curriculum Redesign Committee. Mines.
2023 Graduate Computing Resource Committee (chair). Mines.
2022--present Colloquium Committee. Mines.
2022 CAM Curriculum Redesign Committee. Mines
Organization
2024 Sparse and Low Rank Methods for Unsupervised Learning Session at SIAM MDS.
2024 Differentiating through fixed-points and applications Session at INFORMS IOS.
2022 Learning to Optimize and Optimizing to Learn. Session at SIAM MDS.
2021 Exploiting structure in zeroth-order optimization. Workshop at INFORMS2021.
2020 ZOOM: Zeroth Order Online Meeting. (Online) mini-conference.
Undergrad. Students Mentored
2024 Cash Cherry, Aidan Eck, Sawyer Allen.
2023 Amandin Chyba Rabeendran (Mines –> NYU), Jordan Pettyjohn (Mines).
2021 Yuchen Lou (Hong Kong Univ. –> Northwestern Univ.), Isha Slavin (UCLA –> NYU), Allen Zou (UCSD –> Lawrence Berkeley National Lab.).
2020 Mariam Alaverdian (Los Angeles Community College –> Yale), William Gilroy (Harvey Mudd), Veronica Kirgios (Notre Dame), Carolina Matuk (Univ. Iowa), Tachin Ruangkriengsin (UCLA–> Princeton), Charles Stoksik (UCLA), Chenglin Yang (UCLA –> Columbia), Allen Zou (UCSD).
2018 Lucas Connell (UGA).
Graduate Students Mentored
2024--present Ziyu Li (Mines).
2022--present Antony Sikorski (Mines), Brandon Knutson (Mines)
2019--2022 Howard Heaton (UCLA), Bumsu Kim (UCLA).
Outreach
2020 CSST Mentor. The resulting paper was presented at ICML.
2020 UCLA REU Mentor. The resulting paper was presented at IEEE Big Data 2020
2018 UGA MathCamp. I mentored a group of five high school students on the “Monster Epidemiology” project.
2011--2012 SHAWCO. I tutored students, trained volunteers and co-led the KenSMART project.
Reviewing
2024 ACHA, JMLR, TMLR, AAAI (x3), SIAM SISC (x2), SIAM OPT, EJAM.
2023 AAAI (x3), Neurips (x4).
2022 SIMODS, IEEE TNNLS, NeuRIPS (x2), ICML (x2).
2019 SODA.
Skills
Proficient in Python (specifically: NumPy, SciKitLearn and PyTorch), Matlab, LaTeX, Git, and Markdown. Experienced in mentoring junior researchers, and in explaining technical concepts to non-technical audiences.