Getting started with lsp-mode for Python

Language Server Protocol is a JSON-RPC protocol that allows text editors and IDEs to delegate language-aware features (such as autocomplete, or jump-to-definition) to a common server process. It means that editors can support smart language features just by implementing a generic LSP client.

I'm migrating my Emacs config to use LSP, starting with Python support. It's more powerful than my old setup, and I expect it will only improve over time, as language servers will benefit from more community attention than the long tail of Emacs packages.

Although there was not much configuration required to get LSP working in Emacs, it wasn't always obvious what I needed to do, and I couldn't find many examples of a full Python configuration to get started.

Here are a few questions and issues I encountered.

1. What do I need to install?

The most popular LSP client for Emacs is lsp-mode (although eglot is also in active development). lsp-mode tries to integrate with sensible existing tools to minimise user configuration - it supports popular language servers, and it hooks into Emacs packages like Flycheck and Company.

You will at least want to install lsp-mode, and probably also lsp-ui (which is focused on UI-altering features like popups and "sideline" information).

For Python support, there are two main language servers - pyls and Microsoft Python Language Server. I have used pyls. You have two options for installing pyls and its dependencies:

  1. pip install python-language-server[all]. This will install pyls, and also install its various dependencies that provide particular features: rope for renaming, pyflakes for detecting errors, mccabe for complexity, etc.
  2. pip install python-language-server, and install the dependencies you want directly.

2. Fixing a Jedi compatibility issue with pyls

At time of writing, python-language-server is not compatible with jedi version 0.16. Make sure you adhere to jedi<0.16,>=0.14.1.

3. How do I enable lsp-mode for Python?

For the base lsp-mode, the only required config is to call (lsp) when in python-mode:

(use-package lsp-mode
  :hook
  ((python-mode . lsp)))
(use-package lsp-ui
  :commands lsp-ui-mode)

4. pyls plugins: mypy, isort and black

Some integrations are not available by default in pyls, but are supported by plugins. You can install these with pip install pyls-black pyls-isort pyls-mypy.

To then enable them in lsp-mode, you can use (lsp-register-custom-settings):

(use-package lsp-mode
  :config
  (lsp-register-custom-settings
   '(("pyls.plugins.pyls_mypy.enabled" t t)
     ("pyls.plugins.pyls_mypy.live_mode" nil t)
     ("pyls.plugins.pyls_black.enabled" t t)
     ("pyls.plugins.pyls_isort.enabled" t t)))
  :hook
  ((python-mode . lsp)))

5. Fixing a pyls-mypy issue

At time of writing, the mypy plugin has an issue due to a missing future import. It can be resolved by pip install future. See pyls-mypy #37.

6. What about flake8?

flake8 is not mentioned in the pyls README, but it is supported. There are two options to enable it:

You can use (lsp-register-custom-settings) as before:

(lsp-register-custom-settings
 '(("pyls.plugins.flake8.enabled" t t)))

Alternatively, lsp-mode automatically turns some configuration parameters into custom variables, including the flake8 parameters. So:

(setq lsp-pyls-plugins-flake8-enabled t)

7. What other options does pyls support?

I'm not sure if there is a standard way to retrieve all supported configuration options from a language server. There is a pretty long list of pyls options in the vscode-client.

8. How do I inspect what lsp-mode is doing?

There are a few places you can look for info:

  • The *pyls::stderr* buffer. If something isn't working as expected, this may help identify the problem - eg. it will show issues loading particular plugins.
  • (setq lsp-log-io t) - this will log messages between the lsp client and server to a buffer. You can view the buffer by calling (lsp-workspace-show-log).
  • The lsp-client-settings variable. This contains all the lsp-mode settings for different language servers, and seems to be what you eventually modify when you run (lsp-register-custom-settings).
  • (lsp-describe-session) shows the capabilities of the current session. See the troubleshooting section of the lsp-mode README.

9. How do I support multiple projects?

Python projects often use virtual environments to manage project dependencies. My understanding is that pyls has to be installed in the project's virtualenv, in order to access dependencies for project-aware features like checking symbol references.

Neither lsp-mode nor pyls seem to provide features to manage multiple projects - the appropriate virtualenv needs to be managed separately.

In the past I've used pyvenv for this with some success. For various reasons pyvenv is only able to set a single global virtualenv at a time, but it does have a "tracking" mode, which can automatically change the global virtualenv using dir-locals.

I use a default "emacs" virtualenv as a fallback for editing things like org-mode Python buffers.

(use-package pyvenv
  :demand t
  :config
  (setq pyvenv-workon "emacs")  ; Default venv
  (pyvenv-tracking-mode 1))  ; Automatically use pyvenv-workon via dir-locals

You can use (add-dir-local-variable) to set pyvenv-workon for a particular project.

10. The final code

My initial config for lsp-mode also included a few other settings. It looked roughly like this:

(use-package lsp-mode
  :config
  (setq lsp-idle-delay 0.5
        lsp-enable-symbol-highlighting t
        lsp-enable-snippet nil  ;; Not supported by company capf, which is the recommended company backend
        lsp-pyls-plugins-flake8-enabled t)
  (lsp-register-custom-settings
   '(("pyls.plugins.pyls_mypy.enabled" t t)
     ("pyls.plugins.pyls_mypy.live_mode" nil t)
     ("pyls.plugins.pyls_black.enabled" t t)
     ("pyls.plugins.pyls_isort.enabled" t t)

     ;; Disable these as they're duplicated by flake8
     ("pyls.plugins.pycodestyle.enabled" nil t)
     ("pyls.plugins.mccabe.enabled" nil t)
     ("pyls.plugins.pyflakes.enabled" nil t)))
  :hook
  ((python-mode . lsp)
   (lsp-mode . lsp-enable-which-key-integration))
  :bind (:map evil-normal-state-map
              ("gh" . lsp-describe-thing-at-point)
              :map md/leader-map
              ("Ff" . lsp-format-buffer)
              ("FR" . lsp-rename)))

(use-package lsp-ui
  :config (setq lsp-ui-sideline-show-hover t
                lsp-ui-sideline-delay 0.5
                lsp-ui-doc-delay 5
                lsp-ui-sideline-ignore-duplicates t
                lsp-ui-doc-position 'bottom
                lsp-ui-doc-alignment 'frame
                lsp-ui-doc-header nil
                lsp-ui-doc-include-signature t
                lsp-ui-doc-use-childframe t)
  :commands lsp-ui-mode
  :bind (:map evil-normal-state-map
              ("gd" . lsp-ui-peek-find-definitions)
              ("gr" . lsp-ui-peek-find-references)
              :map md/leader-map
              ("Ni" . lsp-ui-imenu)))

(use-package pyvenv
  :demand t
  :config
  (setq pyvenv-workon "emacs")  ; Default venv
  (pyvenv-tracking-mode 1))  ; Automatically use pyvenv-workon via dir-locals

This is not perfect and will definitely require future work, but it's a useful start. In theory, adding support for a new language should only require installing the language server and adding a couple of lines of elisp to enable the new language.

I'll be updating my config on github.

2020-Apr-26