Getting Started With Magic Functions
If you have any experience with the command line interpreters like BASH, you can utilize some basic CLI command-line calls in Jupyter in the form of predefined functions (“magics”) that utilize command-line-like syntax and are prefaced with % or %%. You can also type %automagic to call magic commands without the %.
Run %magic to get a list of magic functions in Jupyter Notebooks and you’ll get documentation that looks like this:
IPython's 'magic' functions
=========================== The magic function system provides a series of functions which allow you to control the behavior of IPython itself, plus a lot of system-type features. There are two kinds of magics, line-oriented and cell-oriented. ...
One of the most useful magics is %quickref, which brings up a quick reference card for IPython (Jupyter), including information on magic functions:
IPython -- An enhanced Interactive Python - Quick Reference Card ================================================================ obj?, obj?? : Get help, or more help for object (also works as ?obj, ??obj).
?foo.*abc* : List names in 'foo' containing 'abc' in them. ...
If you would like a list of all magics, simply type in %lsmagic and you’ll get this list:
Available line magics:
%alias %alias_magic %autocall %automagic %autosave %bookmark %cd %clear %cls %colors %config %connect_info %copy %ddir %debug %dhist %dirs %doctest_mode %echo %ed %edit %env %gui %hist %history %killbgscripts %ldir %less %load %load_ext %loadpy %logoff %logon %logstart %logstate %logstop %ls %lsmagic %macro %magic %matplotlib %mkdir %more %notebook %page %pastebin %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %popd %pprint %precision %profile %prun %psearch %psource %pushd %pwd %pycat %pylab %qtconsole %quickref %recall %rehashx %reload_ext %ren %rep %rerun %reset %reset_selective %rmdir %run %save %sc %set_env %store %sx %system %tb %time %timeit %unalias %unload_ext %who %who_ls %whos %xdel %xmode Available cell magics:
%%! %%HTML %%SVG %%bash %%capture %%cmd %%debug %%file %%html %%javascript %%js %%latex %%perl %%prun %%pypy %%python %%python2 %%python3 %%ruby %%script %%sh %%svg %%sx %%system %%time %%timeit %%writefile Automagic is ON, % prefix IS NOT needed for line magics.
If you would like to know more about a specific magic function, call the function followed by a question mark, like this: %autocall?. You’ll get documentation for that function:
Docstring:
Make functions callable without having to type parentheses. Usage:
%autocall [mode] The mode can be one of: 0->Off, 1->Smart, 2->Full. If not given, the value is toggled on and off (remembering the previous state). ...
Line Magics
Line magics are defined with % and their arguments only span an single line. For example: %echo "Hello World!" will print out "Hello World!"
Cell Magics
Cell magics are defined using %% and can typically only be used once per cell (though there are some exceptions). They receive an argument from the current line (where they are defined) and from the body of the cell. Here is an example from the documentation:
This magic will create a file called foo.py and write print('hello world') to the file. If you type %ls, you’ll see foo.py in your working directory.
Other Interpreters
You can even use magics to run code in other languages’ interpreters. Using the %ruby magic, we can print to the console:
This will output ‘Hello from Ruby 1.9.3’ (or whichever version you’re using). The same sort of thing can be done using the BASH interpreter:
This will output your BASH filepath, eg. ‘hello from /usr/local/bin/bash’.
You can execute Python files with the .py extension and other Jupyter notebooks using the %run magic. Using a previous example, %run foo.py will print hello world to the console. It’s also possible to write file contents to standard output (ie. the terminal) using %pycat. If you type %pycat foo.py, Jupyter will return the contents of the file we created earlier: print('hello world').
Setting Environment Variables
$env allows you to manage Jupyter Notebook environment variables without having to restart the kernel. If you type %env without an argument, you will get a JSON list of all environment variables. You can set environment variables with %env var=val, where val is the value you want to set the variable to.
Timing
You can time how long a statement or expression takes to execute using %time and the CPU and wall clock times will be printed after execution. %timeit does virtually the same thing, but returns the average of the fastest three times after 100,000 runs.
Shell Commands
These aren’t magic commands, but you can execute shell commands from within Jupyter by using the exclamation mark. For example, !pip install numpy will install numpy. You can also save the output of a shell command to a variable like this: my_shell_path = !echo $SHELL. Bear in mind that not every BASH command will work using !. For more examples, see the documentation.
Defining Your Own Magics
It is possible to define custom magics, either using your own standalone functions or by using a base class from IPython (IPython.core.magic.Magics). This is a bit more advanced feature and requires some understanding of object-oriented programming in Python. If you want to know how to create custom magics, check out the documentation.
Experiment!
This was a very quick introduction to magics and I provided only a few examples, but there are a lot more quite useful magics. The real power of magics shows up when you use them together. You can write SQL queries and R code for different parts of your analysis, and you can even debug using %debug. Play around with magic functions on your own. You’ll be impressed with what you can accomplish.
Links
Python’s Official Documentation for magic functions.
Data analyst Katya Demidova’s excellent and brief overview of some handy magics.
S. John Cruchon-Dupeyrat has provided some great Jupyter cheat sheet with tons of useful magic commands.
Originally published at danstrong.tech.
