nix, shell, perl, php, mysql and mac os x tips and tricks

Friday, March 11, 2016

wget a site for caching purposes (spider it)

Sometimes you want to crawl a site so that it gets cached on its server. This will "spider" a site and not download content
wget -r -l6 --spider -D

1 comment:

  1. If you're in the mood for a fantastic family trip, you won't be disappointed with Gatlinburg and Pigeon Forge, Tennessee. There's plenty to do and see in this mountain community, from award-winning theme parks to world-class performance venues. But best of all, budget-minded vacationers will find a variety of entertainment options guaranteed to keep you smiling even after the credit-card bill arrives. camel spiders bite effects