本课程由黑马程序员精心打造，专为Python爬虫初学者设计，涵盖从基础到高级的全方位内容。课程内容包括爬虫基础、requests模块、数据提取、Selenium、反爬与反反爬、MongoDB数据库、Scrapy框架及其高级应用等。通过实战案例，如模拟登录、数据抓取与分析，学员将掌握如何高效采集和处理网络数据。适合有志于数据科学、网络分析及自动化测试的开发者。立即加入，开启你的Python爬虫之旅！
以下云资源目录树快照生成于[12个月前]，该学习资料由夸克云用户[飞翔*柠檬]分享（只展示大部分文件和目录）
黑马程序员Python爬虫实战：从入门到精通11.75 GB(flv视频132节；png、jpg、gif图片142张；txt文档8个；zip压缩包1个；)
资料
爬虫课件完整版本(HTML版本).zip22.76MB第9节scrapy其他
3.代码
Splash
scrapy.cfg256Byte
baidu_with_splash.html480.25KBbaidu.html227ByteSplash
settings.py3.54KBpipelines.py286Bytemiddlewares.py1.86KBitems.py285Byte__init__.py0Bytespiders
baidu_ws.py619Bytebaidu.py293Byte__init__.py161Byte__pycache__
baidu_ws.cpython-36.pyc971Bytebaidu.cpython-36.pyc635Byte__init__.cpython-36.pyc152Byte__pycache__
settings.cpython-36.pyc620Byte__init__.cpython-36.pyc144Bytedeploy
tencentpro.json532.2KBtencent.json3.01KBpy21.log642Bytelogs
Tencent
tencentpro
a6785c685b4411e9b320186590e02601.log699.84KBtencent
88b639ac5b4411e9ae2d186590e02601.log7.12KBeggs
Tencent
1554868409.egg12.4KBdbs
Tencent.db8KB2.其他资料(空)
1.课件
课件在第7节里.txt0Byte第8节scrapy-redis
3.代码
scrapy-redis
VERSION10Bytetox.ini610ByteTODO.rst767Bytesetup.py1.74KBsetup.cfg76Byterequirements-tests.txt83Byterequirements-setup.txt107Byterequirements-install.txt96Byterequirements-dev.txt139ByteREADME.rst7.16KBpytest.ini154ByteMANIFEST.in227ByteMakefile3.99KBLICENSE1.04KBHISTORY.rst2.55KBCONTRIBUTING.rst3.16KBAUTHORS.rst160Bytetests
test_utils.py213Bytetest_spiders.py4.64KBtest_scrapy_redis.py8.88KBtest_queue.py1.19KBtest_picklecompat.py544Bytetest_package_import.py156Bytetest_dupefilter.py2.61KBtest_connection.py2.14KBsrc
scrapy_redis
utils.py192Bytespiders.py6.67KBscheduler.py5.63KBqueue.py4.32KBpipelines.py2.03KBpicklecompat.py242Bytedupefilter.py4.29KBdefaults.py631Byteconnection.py2.25KB__init__.py197Byteexample-project
scrapy.cfg246Byterequirements.txt20ByteREADME.rst1.54KBprocess_items.py2.87KBDockerfile103Bytedocker-compose.yml142Byteexample
settings.py1.24KBpipelines.py362Byteitems.py613Byte__init__.py0Bytespiders
myspider_redis.py615Bytemycrawler_redis.py871Bytedmoz.py792Byte__init__.py248Byte__pycache__
myspider_redis.cpython-36.pyc987Bytemycrawler_redis.cpython-36.pyc1.17KBdmoz.cpython-36.pyc1.06KB__init__.cpython-36.pyc175Byte__pycache__
settings.cpython-36.pyc658Bytepipelines.cpython-36.pyc566Byte__init__.cpython-36.pyc167Bytedocs
scrapy_redis.rst1.1KBrequirements.txt93Bytereadme.rst27Bytemodules.rst73ByteMakefile6.63KBmake.bat6.32KBinstallation.rst1.01KBindex.rst475Bytehistory.rst28Bytecontributing.rst33Byteconf.py8.17KBauthors.rst28ByteJD
scrapy.cfg248ByteJD
settings.py4.42KBpipelines.py282Bytemiddlewares.py1.86KBitems.py515Byte__init__.py0Bytespiders
book.py3.26KB__init__.py161Byte__pycache__
book.cpython-36.pyc2.37KB__init__.cpython-36.pyc144Byte__pycache__
settings.cpython-36.pyc548Byteitems.cpython-36.pyc503Byte__init__.cpython-36.pyc136Byte2.其他资料(空)
1.课件
课件在第7节里.txt0Byte第7节scrapy基础
3.代码
wangyi
wangyi_simple.json20.48KBwangyi.json54.28KBscrapy.cfg256Bytewangyi
settings.py3.14KBpipelines.py1.58KBmiddlewares.py1.86KBitems.py826Byte__init__.py0Bytespiders
job_simple.py2KBjob_crawl.py1.7KBjob.py2.47KB__init__.py161Byte__pycache__
job_simple.cpython-36.pyc1.36KBjob_crawl.cpython-36.pyc1.33KBjob.cpython-36.pyc1.74KB__init__.cpython-36.pyc174Byte__pycache__
settings.cpython-36.pyc418Bytepipelines.cpython-36.pyc2.39KBitems.cpython-36.pyc795Byte__init__.cpython-36.pyc166Bytemyspider
scrapy.cfg260Byteitcast2.html154.61KBitcast1.html154.61KBitcast.json96.2KBmyspider
settings.py3.12KBpipelines.py708Bytemiddlewares.py196Byteitems.py338Byte__init__.py0Bytespiders
itcast.py2.19KB__init__.py161Byte__pycache__
itcast.cpython-36.pyc912Byte__init__.cpython-36.pyc156Byte__pycache__
settings.cpython-36.pyc326Bytepipelines.cpython-36.pyc891Byteitems.cpython-36.pyc389Byte__init__.cpython-36.pyc148Bytegithub
scrapy.cfg256Bytegit_without_cookies.html99.11KBgit_with_post.html126.79KBgit_with_fromresponse.html126.81KBgit_with_cookies.html126.47KBgithub
settings.py3.07KBpipelines.py286Bytemiddlewares.py1.86KBitems.py285Byte__init__.py0Bytespiders
git3.py1.36KBgit2.py1.34KBgit1.py1.3KB__init__.py161Byte__pycache__
git3.cpython-36.pyc1.11KBgit2.cpython-36.pyc1.45KBgit1.cpython-36.pyc1.81KB__init__.cpython-36.pyc152Byte__pycache__
settings.cpython-36.pyc249Byte__init__.cpython-36.pyc144ByteDouban
scrapy.cfg256ByteDouban
settings.py4.7KBpipelines.py286Bytemiddlewares.py1.13KBitems.py283Byte__init__.py0Bytespiders
movie.py832Byte__init__.py161Byte__pycache__
movie.cpython-36.pyc1002Byte__init__.cpython-36.pyc152Byte__pycache__
settings.cpython-36.pyc1.89KBmiddlewares.cpython-36.pyc1.15KBitems.cpython-36.pyc350Byte__init__.cpython-36.pyc144ByteAQI
scrapy.cfg250ByteAQI
settings.pyc663Bytesettings.py3.08KBpipelines.py283Bytemiddlewares.pyc2.5KBmiddlewares.py720Byteitems.pyc652Byteitems.py628Byte__init__.pyc136Byte__init__.py0Bytespiders
aqi.pyc2.67KBaqi.py2.48KB__init__.pyc144Byte__init__.py161Byte__pycache__
aqi.cpython-36.pyc1.86KB__init__.cpython-36.pyc147Byte__pycache__
settings.cpython-36.pyc313Bytemiddlewares.cpython-36.pyc833Byteitems.cpython-36.pyc515Byte__init__.cpython-36.pyc139Byte2.其他资料(空)
1.课件
README.md655Byte9.scrapy_splash组件的使用.md6.23KB8.scrapy_redis原理分析并实现断点续爬以及分布式爬虫.md6.38KB7.scrapy_redis概念作用和流程.md1.99KB6.scrapy中间件的使用.md8.71KB5.scrapy管道的使用.md4.11KB4.scrapy模拟登陆.md5.12KB3.scrapy构造并发送请求.md8.65KB2.scrapy的入门使用.md7.02KB14.crawlspider类的使用.md5.28KB13.scrapy总结图.md64Byte13.gerapy爬虫管理.md3.83KB12.gerapy爬虫管理.md3.96KB11.scrapyd部署scrapy项目.md3.34KB10.scrapy的日志信息与配置.md6.86KB1.scrapy的概念作用和工作流程.md3.49KBimages
运行2.png140.17KB运行1.png92.45KB部署成功.png164.78KB部署.png228.5KBproject项目打包.png252.98KBproject_list.png92.83KBproject_1.png26.49KBgerapy_主界面.png33.44KBgerapy_主机添加.png25.48KBgerapy_主机管理页面_添加.png85.52KBgerapy_主机管理页面.png79.98KBgerapy_调度scrapy爬虫项目.png41.18KBgerapy_数据库初始化.png322.63KBgerapy_配置scrapyd服务器.png33.2KBgerapy_目录结构.png25.09KBgerapy_列表.png37.16KBbuild之后.png263.43KB9.4.5.2.with-splash.png88.64KB9.4.5.2.no-splash.png39.6KB9.3.1.3.splash-server.png86.62KB8.4.3.scheduler.png293.35KB8.4.2.RFP.png385.96KB8.4.1.redis_pipeline.png163.34KB8.3.domz运行现象.png415.17KB7.4.2.scrapy_redis的流程.png374.25KB
4.3.1request-payload.jpg8.34KB3.1.scrapy翻页.png13.46KB2.2.scrapy入门使用-2.png188.52KB2.1.scrapy入门使用-1.png155.44KB12.scrapy总结图.png528.16KB11.scrapyd-7.jpg106.92KB11.scrapyd-6.jpg53.08KB11.scrapyd-5.jpg62.47KB11.scrapyd-4.jpg61.67KB11.scrapyd-3.jpg49.95KB11.scrapyd-2.jpg42KB11.scrapyd-1.jpg128.38KB10.1.scrapy_debug.png347.62KB1.3.4.scrapy组件.png340.79KB1.3.3.scrapy工作流程.png217.8KB1.3.2.爬虫流程-2.png101.56KB1.3.1.爬虫流程-1.png66.62KB0.jpg28.12KB第6节MongoDB数据库
3.代码
1.pymongo_test.py853Byte2.其他资料(空)
1.课件
README.md525Byte8.mongodb总结图.md61Byte7.mongodb和python交互.md4.45KB6.mongodb的权限管理.md2.98KB5.mongodb的索引操作.md3.1KB4.mongodb的聚合操作.md7.09KB3.mongodb的增删改查.md5.89KB2.mongodb的简单使用.md4.44KB1.mongodb介绍和安装.md2.97KBimages
mongodb的聚合.png188.96KB7.mongodb总结.png175.69KB4.3.创建索引速度对比.png198.3KB第5节反爬与反反爬
3.代码
yundama.py4.69KBYDMHTTPDemo3.x.py4.62KB1.pytessseract.py174Byte2.其他资料(空)
1.课件
README.md82Byte4.JS的解析.md7.87KB3.chrome在爬虫中的使用.md3.37KB2.打码平台的使用.md9.72KB1.常见的反爬手段和解决方法.md10.8KBimages
新建隐身窗口.png87.85KB通过css反爬.png226.41KB添加断点.png386.78KB使用字体来反爬.png706.72KB人人网抓包寻找url地址.png305.69KB人人网手机版登录.png292.05KB人人网登录的地址.png254.61KB继续在其中搜索关键字.png109.64KB勾选perservelog的效果.png224.52KB点击美化输出.png68.15KBsearchallfile搜索关键字.png373.29KBjs的绑定事件.png303.47KBchrome中network更多用途.png108.11KB第4节selenium
3.代码
9.execute_js.py251Byte8.selenium_cookies.py240Byte7.qzone.py480Byte6.window_handler.py803Byte5.58.py555Byte4.locate_test.py870Byte3.selenium_attribute&func.py480Byte2.selenium_test.py389Byte12.douyu.py1.68KB11.selenium_config.py608Byte10.wait.py206Byte1.example_test.py1003Byte2.其他资料(空)
浏览器插件
Windows
XPath-Helper_v2.0.2.crx246.57KBSwitchyOmega-2.3.21.crx777.91KBJSONView0.0.32.2_0.crx94.08KBMacOSX
XPath-Helper_v2.0.2.crx246.57KBSwitchyOmega-2.3.21.crx777.91KBJSONView0.0.32.2_0.crx94.08KB1.课件
README.md285Byte3.selenium的其它使用方法.md11.62KB2.selenium定位获取标签对象并提取数据.md3.16KB1.selenium的介绍.md5.84KBimages
下载chromedriver-4.png74.15KB下载chromedriver-3.png66.08KB下载chromedriver-2.png72.65KB下载chromedriver-1.png129.67KB查看chrome版本2.png46.82KB查看chrome版本.png75.83KBselenium的工作原理.png22.48KB第3节数据提取
3.代码
lagou.py401Byte9.teiba.py2.46KB8.lxml_text.py1.26KB6.path_test.py896Byte5.jsonpath_test.py352Byte2.其他资料(空)
1.课件
README.md262Byte3.数据提取-lxml模块.md12.54KB2.数据提取-jsonpath模块.md2.84KB1.数据提取概述.md3KBimages
数据解析.png164.29KB结构化响应内容json.png477.69KB节点.png136.31KB非结构化响应内容html.png457.69KBxpath中节点的关系.png244.3KBxpath_helper插件安装-3.png353.12KBxpath_helper插件安装-2.png46.27KBxpath_helper插件安装-1.png87.49KB
xml树结构.gif3.76KBxml和html的区别.png93.36KBjsonpath使用示例.png33.04KBjsonpath的方法.png148.93KB第2节requests模块
3.代码
9.ca.py149Byte8.reqeusts_proxy.py405Byte7.requests_cookies.py586Byte6.requests_cookies.py1.25KB5.requests_headers_cookies.py1.02KB4.requests_params.py751Byte3.zhihu.py404Byte2.reqeusts_headers.py459Byte11.requests_session.py684Byte10.king.py1.17KB1.easy_requersts.py623Byte2.其他资料(空)
1.课件
requests模块.md22.64KBREADME.md434Byteimages
使用代理的过程.png100.35KB查看移动端页面.png293.66KB百度翻译移动版-4.png161.37KB百度翻译移动版-3.png174.03KB百度翻译移动版-2.png169.31KB百度翻译移动版-1.png240.86KB
responsedata.jpeg240.34KBrequests-3-2-3.png45.63KBrequests-3-2-1-4.png194.32KBking.jpeg219.67KBformdata.jpeg20.88KB4-2-2-3.jpg112.22KB4-2-2-2.jpg22.71KB4-2-2-1.jpg82.96KB12306ssl错误.png256.58KB第1节爬虫基础
3.代码(空)
2.其他资料(空)
1.课件
README.md276Byte2.http协议复习.md5.74KB1.爬虫概述.md3.46KBimages
爬虫分类.png170.01KB爬虫的工作流程.png87.95KBhttp作用.png96.06KBhttp发送的过程.png211.39KBhttp的响应形式.png80.99KBhttp的请求形式.png61.49KBcontent-type.webp9.71KB第10节appium
3.代码
douyin.py3.58KB2.其他资料(空)
1.课件
README.md308Byte2.利用appium自动控制移动设备并提取数据.md8.42KB1.appium环境安装.md5.7KBimages
元素定位.png977.23KBjava-jdk1.8下载.png149.61KBappium-session配置.png62.79KBandroid-sdk-manager更新-2.png50.19KBandroid-sdk-manager更新.png24KB2-3-2-appiumDesktop定位界面使用方法.jpg42.36KB2.2.3.6.png76.83KB2.2.3.5.png150.62KB2.2.3.4_.png63.52KB2.2.3.3_.png16.19KB2.2.3.2_.png27.12KB2.2.3.1_.png56.42KB2.2.2.1.png13.6KB2.2.1.2.png121.38KB2.2.1.1.png85.4KB2.1.5.4.jpg54.44KB2.1.5.3.jpg21.2KB2.1.5.2.jpg60.79KB2.1.5.1.jpg45.9KB2.1.4.3.jpg25.94KB2.1.4.2.jpg25.65KB2.1.4.1.jpg41.78KB2.1.3.9.jpg42.56KB2.1.3.8.jpg48.14KB2.1.3.7.jpg29.05KB2.1.3.6.jpg105.76KB2.1.3.5.jpg33.81KB2.1.3.4.jpg36.19KB2.1.3.3.jpg35.64KB2.1.3.2.jpg25.88KB2.1.3.15.jpg48.33KB2.1.3.14.jpg60.97KB2.1.3.13.jpg63.13KB2.1.3.12.jpg36.55KB2.1.3.11.jpg42.63KB2.1.3.10.jpg8.86KB2.1.3.1.jpg51.06KB2.1.2.8-安装java.png61.75KB2.1.2.7-安装java.png30.24KB2.1.2.6-安装java.jpg57.25KB2.1.2.5-安装java.jpg40.05KB2.1.2.4-安装java.jpg41.89KB2.1.2.3-安装java.jpg25.69KB2.1.2.2-安装java.jpg49.12KB2.1.2.1-安装java.jpg106.87KB2.1.1.1安装node.js.png73.87KB10_appium
04.appium使用.flv156.94MB03.appium环境联调测试.flv120.93MB02.appium环境搭建.flv288.17MB01.appium简介.flv48.25MB09_scrapy其他
07.使用Gerapy进行爬虫管理.flv104.04MB06.使用scrapyd进行爬虫部署.flv130.01MB05.scrapy配置-其他配置.flv69.76MB04.scrpay配置-常用配置.flv127.16MB03.scrapy-日志信息.flv142.72MB02.splash-使用.flv118.54MB01.splash-介绍_环境配置.flv111.77MB08_scrapy-redis
10.scrapy-redis-京东爬虫-修改成分布式爬虫.flv236.06MB09.scrapy-redis-京东爬虫-图书价格.flv216.34MB08.scrapy-redis-京东爬虫-图书列表页面.flv197.01MB07.scrapy-redis-京东爬虫-图书分类页面.flv199MB06.scrapy-redis-京东爬虫-简介.flv187.43MB05.scrapy-redis-分布式爬虫编写流程.flv38.27MB04.scrapy-redis-分布式爬虫实现.flv285.84MB03.scrapy-redis-源码分析.flv342.33MB02.scrapy-redis-断点续爬.flv243.31MB01.scrapy-redis-分布式的概念_原理.flv169.73MB07_scrapy基础
23.中间件-selenium动态加载.flv282.67MB22.中间件-随机ip代理.flv204.38MB21.中间件-随机请求头.flv115MB20.中间件-豆瓣爬虫.flv169.93MB19.中间件-分类与作用.flv95.57MB18.tencent_crawl.flv55.13MB17.crawlspider.flv39.12MB16.scrapy管道的使用.flv64.8MB15.scrapy模拟登陆-FormRequest发送post请求.flv34.33MB14.scrapy模拟登陆-cookies参数使用.flv31.44MB13.请求对象-meta传参.flv29.64MB12.请求对象-其他参数.flv14.78MB11.请求对象-案例-网易招聘.flv55.8MB10.请求对象-请求&网易招聘简介.flv7.28MB09.项目开发流程-数据建模&流程完善.flv22.04MB08.项目开发流程-5保存数据.flv122.59MB07.项目开发流程-4完成爬虫.flv61.85MB06.项目开发流程-3创建爬虫.flv25.12MB05.项目开发流程-2创建项目.flv8.92MB04.项目开发流程-1框架安装&流程简介.flv3.86MB03.scrapy-各模块功能.flv10.91MB02.scrapy-工作流程.flv32.3MB01.scrapy-概念&作用.flv6.43MB06_MongoDB数据库
17.pymongo使用.flv34.81MB16.mongodb权限管理-普通用户.flv13.7MB15.mongodb的权限管理-管理用户.flv10.35MB14.mongodb-索引.flv34.72MB13.聚合管道-其他管道.flv30.12MB12.聚合管道-分组管道.flv29.74MB11.mongodb-修改&删除.flv29.66MB10.mongodb查询结果操作.flv19.83MB09.mongodb查询数据.flv26.28MB08.mongdb插入数据.flv16.25MB07.mongodb的数据类型.flv12.74MB06.mongdb简单使用-集合操作.flv19.66MB05.mongodb简单使用-数据库操作.flv7.88MB04.mongodb简单使用-生产启动.flv91.64MB03.mongodb简单使用-测试启动.flv47.13MB02.mongodb的安装.flv38.24MB01.mongodb的介绍.flv43.03MB05_反爬与反反爬
18.有道翻译实现.flv128.38MB17.hashlib使用.flv120.98MB16.有道翻译分析.flv93.4MB15.js解析-人人网登陆.flv31.05MB14.js解析-js2py使用.flv87.64MB13.js解析-js代码分析.flv93.38MB12.js解析-定位js文件.flv73.71MB11.js解析-人人网登录分析.flv74.38MB10.chrome浏览器的使用.flv85.38MB09.验证码-打码平台.flv125.58MB08.验证码-图像识别引擎.flv56.1MB07.验证码-验证码的知识.flv27.34MB06.反爬与反反爬-基于数据加密进行反爬.flv130.21MB05.反爬与反反爬-及与爬虫行为的反爬.flv120.16MB04.反爬与反反爬-基于身份识别的反爬.flv39.78MB03.反爬与反反爬-常见概念与反爬方向.flv26.8MB02.反爬与反反爬-什么样的爬虫会被反.flv19.25MB01.反爬与反反爬-反爬原因.flv18.63MB04_selenium
14.selenium-案例-斗鱼直播.flv568.09MB13.selenium-配置对象.flv27.08MB12.selenium-页面等待.flv77.65MB11.selenium-执行js.flv119.77MB10.selenium-cookies操作.flv64.44MB09.selenium-窗口切换.flv125.82MB08.selenium-标签切换.flv111.81MB07.selenium-元素操作.flv54.51MB06.selenium-元素定位.flv184.14MB05.selenium-driver属性和方法.flv27.54MB04.selenium-简单使用.flv36.23MB03.selenium-模块与driver的安装.flv98.76MB02.selenium的工作原理.flv21.9MB01.selenium-介绍_效果展示.flv87.91MB03_数据提取
14.tostring.flv17.23MB13.百度贴吧.flv90.71MB12.lxml使用.flv137.21MB11.xpath语法-其他常用语法.flv16.5MB10.xpath语法-节点修饰语法.flv458.13MB09.xpath语法-基础节点选择.flv149.41MB08.xpathhelper简介_安装.flv64.4MB07.lxml简介.flv29.75MB06.jsonpath-案例-拉钩.flv35.99MB05.jsonpath-练习.flv29.55MB04.jsonpath-场景-安装-使用.flv54.77MB03.常用数据解析方法.flv18.33MB02.xml与html.flv22.7MB01.数据提取-响应内容的分类.flv168.45MB02_requests模块
9.requests模块-timeout参数.flv32.52MB8.requests模块-cookiejar对象的处理.flv26.16MB7.requests模块-cookies参数的使用.flv89.58MB6.requests模块-在headers中设置cookies参数.flv92.53MB5.requests模块-发送带参数的请求.flv107.85MB4.requests模块-发送带请求头的请求.flv92.94MB3.requests模块-响应对象常用的参数和方法.flv39.53MB2.requests模块-响应对象的text与content之间的区别.flv49.57MB15.案例-github模拟登陆.flv270.08MB14.requests模块-session.flv19.82MB13.requests-post数据来源.flv199.73MB12.requests模块-post请求_金山翻译案例.flv170.88MB11.requests模块-verify参数与ca证书.flv44.06MB10.requests模块-代理proxies使用.flv128.07MB1.requests模块-简介-安装-使用.flv52.22MB01_爬虫基础
10.浏览器运行过程.flv148.39MB09.创建的响应状态码.flv75.57MB08.常见的请求头与响应头.flv164.32MB07.http与https的区别.flv27.64MB06.爬虫的流程.flv46.18MB05.爬虫的分类.flv68.23MB04.爬虫的作用.flv178.67MB03.爬虫的概念.flv32.66MB02.课程结构.flv12.74MB01.爬虫特点介绍.flv35.35MB
