Update on Technical Details

Three months ago, during an extended break in the monsoon, I wrote here Technical Details As Acknowledgements. Much of that post was about the process of installing free software, which I've been using for a variety of things, including producing custom weather maps on my MacBook. One motivation for providing information about the process was that lessons learned might be helpful for anyone interested in doing something similar with various Python packages. Another goal was to acknowledge the effort that people put into making these software packages work and remain freely available.

Most of the details from three months ago remain relevant. A few things have changed, and I'll incorporate those changes into a description of the process starting from scratch on a new MacBook. I had been limping along on a MacBook Air that I bought almost eight years ago, cheaply at the time as a refurbished unit. But recently I succumbed to temptation and bought a new one, which is equipped with a processor chip in the Apple M1 family, known as Apple Silicon, but also as ARM to distinguish from Intel.

I had been reading online discussions about M1, and based on advice in those discussions I expected for now to be able to get only so far with installing things natively on the new Macbook; for some things I would need to keep using my old Macbook. Some online posts advised running Intel versions of software through Apple's Rosetta 2 transition program on the M1 machines. Some recommended that the only hope for doing things natively was to use one of the Conda packages. Those other online recommendations may have been true a few months ago, but it is not so now. Everything that I had been using is now running natively on my new M1 Macbook, with no need for a middle-man software manager like miniconda. (Homebrew is of course a middle-man, but it is much closer to do-it-yourself.) It's time to shut down the eight-year-old Macbook for good.

I have my apple ID registered as a developer. It doesn't cost anything to do just that. That allows me to download the latest versions of Xcode and the Command Line Tools from http://developer.apple.com/downloads/more. Xcode downloads as an .xip file, and the CLT as a .dmg file. To start the install process for either one, double click. It's my understanding that only the CLT are needed by Homebrew. But I always download the latest version of Xcode as well because I need it for a few small stand-alone programs. I always do a separate install of the CLT after installing Xcode, and that seems to ensure that the Homebrew formulas find the CLT in the expected place.

Once again here is the Homebrew installation instruction web page. After completing those instructions on the M1 Macbook, I did brew install gcc and then brew install python@3.9. If either of those failed, there would be no point in trying to go farther. But everything installed natively and automatically with no problems, including several dependencies for each. I then did brew install geos and brew install proj. Those two libraries are needed by the cartopy python package. Three months ago cartopy required an older version of proj, but now cartopy uses the current version. There are a couple of dependencies automatically installed with proj. Everything continued to install natively and smoothly.

A few python packages can be installed either with a Homebrew formula or with the ordinary Python package manager. That was the case several years ago for numpy and scipy, and for awhile I kept them updated with their Homebrew formulas. Then it seemed that these formulas were unsupported, and the recommendation was to install/update numpy and scipy as ordinary Python packages. So I had been doing that recently. But in the online discussions about M1 there were reports of problems with installing these packages, and the recommendation was to use the Homebrew formulas, which have been updated recently. So brew install numpy and then brew install scipy. Again there are several dependencies, and again everything installed natively and smoothly.

Just a few more manual brew install's: hdf5 netcdf eccodes and pkg-config. Then it's on to the python package manager, already installed as pip3 by the Homebrew python@3.9 formula. There is a deprecation warning that is printed with each package installed. It appears that Homebrew will have to change something in the future, but the warning can be ignored for now. I started with pip3 install matplotlib, which automatically installs a number of required packages. Most if not all of these appear to install as native, pre-compiled binary wheels. Then on to
pip3 install shapely --no-binary shapely
pip3 install pyshp
pip3 install pyproj. These three are from the cartopy installation page. Then pip3 install cartopy.

One of the python packages that can be installed with a Homebrew formula is ipython. But the latest version of python, 3.10, was released just last month. The Homebrew formula for ipython is already set to require that recently released version of python as a dependency, while many other packages still depend on python 3.9. So it's easier to just use the alternative, pip3 install ipython. That also installs a number of dependencies, and they all go into Homebrew's 3.9 site packages folder.

The pandas package installed with no problems with pip3, though it took a long time to compile. Then pip3 install cfgrib, pip3 install xarray and pip3 install MetPy, and back in business plotting grib files on my new Macbook.