Skip to content

2025

  • in ANN

Achieving Scalable Performance on Commodity Hardware for Large Model Training

Training large-scale artificial intelligence models has become a defining challenge of the modern computational era, often perceived as a domain exclusive to those with access to state-of-the-art, unencumbered supercomputing infrastructure. However, recent advancements have demonstrated that exceptional performance can be achieved even with resource constraints and heterogeneous or restricted hardware. This is accomplished through a sophisticated synthesis of software optimization, algorithmic innovation, and a deep understanding of the underlying system architecture. This post delves into the technical strategies that enable high-throughput training, focusing on innovations in parallelism, communication, and low-precision arithmetic.

  • in PINN

Enhancing Parametric PINN Training: A Synergistic Approach with Weighted Curricula and Learning Rate Scheduling

Physics-Informed Neural Networks (PINNs) have emerged as a powerful paradigm for solving differential equations by embedding physical laws directly into the loss function of a neural network. A particularly compelling application is in solving parametric inverse problems, where the goal is to infer physical parameters (e.g., viscosity, conductivity) from observational data. A common and effective methodology involves a two-stage process: first, training a general parametric model over a range of the parameter, and second, using this pre-trained model as a prior to rapidly identify the specific parameter value that best explains a new set of observations.

  • in PINN

Overcoming Generalization Challenges in 2D Burgers' Equation with Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have emerged as a powerful paradigm in "scientific machine learning," enabling the integration of physical laws, expressed as Partial Differential Equations (PDEs), into neural network training (Wang et al., 2021, p. 1; Lu et al., 2021, p. 208). While PINNs offer a promising avenue for solving complex, non-linear systems like the 2D Burgers' equation, ensuring robust generalization remains a critical challenge, particularly for solutions exhibiting multi-scale features, anisotropic behavior, or in data-poor environments. The 2D Burgers' equation is a fundamental non-linear PDE often used as a benchmark for such challenges (Raissi et al., 2019, p. 686). This post explores several cutting-edge approaches aimed at enhancing PINN generalization capabilities, directly addressing the underlying limitations that hinder their performance.

  • in PINN

Overcoming Challenges in Physics-Informed Neural Networks (PINNs): Gradient Optimization for Inverse Problems

Physics-Informed Neural Networks (PINNs) represent a promising approach for solving complex Partial Differential Equations (PDEs) and inverse problems, such as determining hidden parameters of a physical system – for instance, viscosity (nu) in a 2D Burgers equation. However, the application of PINNs presents inherent challenges. A recent study by Wang et al. (2020) delves into these limitations and proposes innovative solutions that can significantly enhance PINN performance.

  • in Tools

Marimo

Interesting alternative to Jupyter Notebook, but with other goals and features. The idea is to embed everything inside a single .py file, instead of an .ipynb file. The .py file is used both for the graphical interface (interactive web app) and for command line execution. It tries to eliminate some of Jupyter's reproducibility issues. It has features such as integration with package and project managers, such as uv. It's worth checking out.

https://marimo.io/