Consult-web: A package to get web and omni search results directly in emacs minibuffer!

consult-web-sc

My new package, consult-web allows you to get search results from both online sources (google, bing, brave, wikipedia, …) and local sources (buffers, org notes, elfeed feeds, gptel ai chatbots, …) directly in minibuffer. It provides similar functionality to consult-buffer but on steroid! It uses consult, url-retrieve and API services form online sources.

GitHub Repo: https://github.com/armindarvish/consult-web

For detailed demonstration of features, comparison with other packages, and usage examples see the YouTube video and the blog post below.

YouTube Video: https://www.youtube.com/watch?v=7pDfyqBZwvo
Blog Post: https://www.armindarvish.com/post/web_omni_search_in_emacs_with_consult-web/

4 Likes

Fantastic package! Thank you!
It’s already useful to me, even though I have only figured out how to set up half of what I’d like to use it for. A damn shame that most search providers don’t let you search anonymously via API when it’s perfectly possible to do so via browser. If ddg, google and bing worked like wikipedia, this would be a dream come true!

I didn’t quite understand how to set up the consult-notes sources. I assumed that I should replace the “Reference Roam Nodes” and “Zettel Roam Nodes” with the entries I configured in consult-notes-file-dir-sources. But that didn’t work. Maybe because I use it with denote and not org-roam?

One idea for academic uses that immediately came to mind was to search through a bibtex file or leverage citar to do so.

But I’m starting to ramble. Thank you so much for this package!

@jabbo Thanks!

I didn’t quite understand how to set up the consult-notes sources. I assumed that I should replace the “Reference Roam Nodes” and “Zettel Roam Nodes” with the entries I configured in consult-notes-file-dir-sources. But that didn’t work. Maybe because I use it with denote and not org-roam?

I do not use denote, so I don’t have it in the consult-notes sources. I should look into that. In the meanhile take a look at:
consult-web/sources/consult-web-notes.el at 0e065aeebac37bfaf307aa2b5c8aea4c83ce52ea · armindarvish/consult-web · GitHub

There is a macro consult-web--make-source-from-consult-source that should allow turning your consult-notes denote source to a consult-web compatible source, but you still need to figure out how to define the preview and callback functions if needed!

One idea for academic uses that immediately came to mind was to search through a bibtex file or leverage citar to do so.

Take a look at my blog post here: https://www.armindarvish.com/post/web_omni_search_in_emacs_with_consult-web/

I show how to use zotra and then pass things to citar, etc. I think all of that stuff is better done outside consult-wb, you just need to make a pipeline to integrate them together.

Lastly since you said

A damn shame that most search providers don’t let you search anonymously via API when it’s perfectly possible to do so via browser.

There is a way to get anonymous results by parsing the http response similar to what some other tools/packages do. I won’t be directly supporting this because technically this would be violating terms of some of those services and I do not want to promote this. But I do pan to make some wiki page demonstration of parsing http response in general for example on a public database like PubMed, which then you can use to parse any webpage and get the results in consult-web if you decide to do so!

1 Like

@daviwil I think you should do another emacs episode specifically for consult and embark (either a stream or one of you in-depth emacs reviews) focusing on workflows (getting items from consult and doing embark actions on them). I think people would be interested in seeing some interesting workflows especially with packages like my consult-web and consult-gh. But there are many of these consult-* packages and commands and personally I would love to see one of your nicely organized reviews covering all the useful consult-* packages/commands and how to put them to good use along with embark actions especially for coding and reviewing other packages, … I am also interested in reducing my time in the browser and porting things I do in the browser to emacs, and I think consult-* and embark can help there too.

By the way, is there a good way for me to write this comment here from inside emacs. The editor in the browser is just killing me!

1 Like