SpiderFoot: Your Ultimate OSINT Companion

According its Github description,

SpiderFoot is an open source intelligence (OSINT) automation tool. It integrates with just about every data source available and utilises a range of methods for data analysis, making that data easy to navigate.
SpiderFoot has an embedded web-server for providing a clean and intuitive web-based interface but can also be used completely via the command-line.

https://github.com/smicallef/spiderfoot

Uses

SpiderFoot is a versatile tool that serves both offensive and defensive purposes. It can be employed offensively in activities like red team exercises or penetration tests to gather reconnaissance data on your target. In the same way, it can be used defensively to assess what information you or your organization might inadvertently reveal on the Internet.

SpiderFoot is a handy tool with more than 200 modules that can do many things, such as:

  • Finding and listing website addresses and their sub-parts.
  • Getting email addresses, phone numbers, and people’s names.
  • Spotting Bitcoin and Ethereum digital money addresses.
  • Checking if someone could steal a sub-part of a website.
  • Looking at all the info stored in a website’s address records.
  • Asking questions to know if something is dangerous or blacklisted.
  • Connecting easily with other online tools like SHODAN, HaveIBeenPwned, GreyNoise, AlienVault, SecurityTrails, and more.
  • Making a list of social media accounts.
  • Checking cloud storage areas like S3, Azure, and DigitalOcean, even grabbing data from them.
  • Figuring out where in the world an internet address is.
  • Reading and understanding things on websites, even picking out specific information.
  • Finding extra info hidden inside files, like pictures, documents, or other computer stuff.
  • Going deep into hidden parts of the internet.
  • Checking which computer doors are open and what services are behind them.
  • Looking for times when important data has been accidentally made public.
  • And many more useful things!

In simple terms, SpiderFoot is like a Swiss Army knife for finding information on the internet, making it a valuable tool for various online tasks.

Installation

Blackarch

$ yay -Ss spiderfoot

blackarch/spiderfoot 4.0-2 (2.9 MiB 14.5 MiB) [blackarch blackarch-recon]    
   The Open Source Footprinting Tool.  
aur/spiderfoot 4.0-1 (+4 0.00)    
   SpiderFoot is an open source intelligence automation tool. Its goal is to automate the process of gathering intelligence about a given target.

$ yay -S blackarch/spiderfoot

buscando conflictos entre paquetes...  
  
Paquete (46)                       Versión nueva      Diferencia neta  Tamaño de la descarga  
  
extra/mypy                         1.3.0-1                  20,11 MiB               3,04 MiB  
blackarch/python-adblockparser     0.7-4                     0,07 MiB               0,02 MiB  
extra/python-astroid               2.15.6-1                  3,45 MiB               0,49 MiB  
extra/python-cheroot               9.0.0-5                   0,56 MiB               0,11 MiB  
extra/python-cherrypy              18.8.0-3                  3,58 MiB               0,56 MiB  
blackarch/python-cherrypy-cors     1.6-5                     0,03 MiB               0,01 MiB  
...
extra/python-stem                  1.8.2-1                   4,49 MiB               0,66 MiB  
extra/python-tempora               5.2.2-2                   0,13 MiB               0,03 MiB  
extra/python-toml                  0.10.2-9                  0,20 MiB               0,04 MiB  
extra/python-wrapt                 1.14.1-2                  0,25 MiB               0,05 MiB  
extra/python-zc.lockfile           3.0.post1-2               0,05 MiB               0,01 MiB  
blackarch/spiderfoot               4.0-2                    14,54 MiB               2,87 MiB  
  
Tamaño total de la descarga:      60,61 MiB  
Tamaño total de la instalación:  407,44 MiB  
  
:: ¿Continuar con la instalación? [S/n]    
:: Obteniendo los paquetes...  
python-matplotlib-3.7.2-1-x86_64                                                     5,7 MiB  2,03 MiB/s 00:03 [------------------------------------------------------------------] 100%  
python-pandas-1.5.3-3-x86_64                                                        12,3 MiB  4,37 MiB/s 00:03 [------------------------------------------------------------------] 100%  
...
(46/46) instalando spiderfoot                                                                                   [------------------------------------------------------------------] 100%  
:: Ejecutando los «hooks» de posinstalación...  
(1/1) Arming ConditionNeedsUpdate...

Other linux distros

 wget https://github.com/smicallef/spiderfoot/archive/v4.0.tar.gz
 tar zxvf v4.0.tar.gz
 cd spiderfoot-4.0
 pip3 install -r requirements.txt

Running

After a default’s installation, Spiderfoot is installed in this folder: /usr/share/spiderfoot
(Using Blackarch installation option)

We can add this folder to the PATH and then just run:
export PATH=/usr/share/spiderfoot:$PATH
$source .bashrc

$ ./sf.py -l 127.0.0.1:5001

*************************************************************  
Use SpiderFoot by starting your web browser of choice and    
browse to http://127.0.0.1:5001/  
*************************************************************  
  
2023-09-16 13:14:18,851 [INFO] sf : Starting web server at 127.0.0.1:5001 ...  
2023-09-16 13:14:18,858 [WARNING] sf :    
********************************************************************  
Warning: passwd file contains no passwords. Authentication disabled.  
Please consider adding authentication to protect this instance!  
Refer to https://www.spiderfoot.net/documentation/#security.  
********************************************************************