1 Star 0 Fork 43

lidong1266/simspider

forked from calvinwilliams/simspider 
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
克隆/下载
ChangeLog-EN 3.11 KB
一键复制 编辑 原始数据 按行查看 历史
calvinwilliams 提交于 2015-03-24 18:52 . UPDATE TO V2.5.3
ver 2.5.3 2015-03-24 calvin
* Fixed a problem about not support CURLOPT_ACCEPT_ENCODING and CURLOPT_TRANSFER_ENCODING on low-version libcurl by use condition macro
* Fixed a bug in function ResetSimSpiderEnv with multiplexing concurrency
ver 2.5.2 2015-03-23 calvin
* Add function SetMaxRetryCount for setting max retry count
* Add function GetDoneQueueUnitRetryCount for getting retry count at last
* Add function SetAcceptEncoding for setting http accept-encoding
* Add function SetTransferEncoding for setting http transfer-encoding
* Add libcurl's DNS Cache
* Swap HtmlLinkParser and pfuncResponseBodyProc
* Improve the main process framework of events for performance
ver 2.5.1 2015-03-16 calvin
* Solve a problem of compilation
* Add function 'MoveUnsuccessfulDoneQueueUnitsToRequestQueue','ResetDoneQueue' used to move back to request queue from done queue
ver 2.5.0 2015-03-14 calvin
* Adjusted function FuncTravelDoneQueueProc prototype
* Add request queue and queue callback mechanism, the user can alternative the two queue using database or Redis and so on ( experimental )
* Fixed a problem with url contains multibyte-characters
ver 2.4.0 2015-03-13 calvin
* Add multiplexing concurrency automatic adjustment mechanism
* Add HTTP request header custom special list
* Append request header 'Referer' automatically
* Improve the main process framework of events for performance
* The curl object reuse mechanism for performance
* Adjust the sample crawler 'simspider' code
ver 2.3.0 2015-03-08 calvin
* Add the error code SIMSPIDER_ERROR_FUNCPROC_INTERRUPT, if callback function returns the code, crawling process interrupt immediately
* Modify the function AppendRequestQueue prototypes
* Modify the function SimSpiderGo prototypes
* Add function SetSimSpiderPublicDataPtrGetSimSpiderPublicDataPtr , Used to pass the crawler engine public data
* Add function etDoneQueueUnitPrivateDataPtrGetDoneQueueUnitPrivateDataPtr , Used to pass private data when crawl each url
* Add function EnableHtmlLinkerParser , used to set the crawler engine internal quick HTML links parser enabled, the default is enabled
* Fixed a concurrent problem
* Optimized the crawler example program 'simspider'
ver 2.2.1 2015-02-09 calvin
* Fixed the automatic completion bug from entrance to website
* Fixed a safe pointer bug
* Fixed a bug on WINDOWS platform
ver 2.2.0 2015-02-07 calvin
* Modify the function SimSpiderGo prototypes. Well, I don't change since
* Modify the function name AppendRequestUnit to AppendRequestQueue. Well, I don't change since too
* add function ResizeRequestQueue used to adjust the request queue size, in order to adapt to crawl
* Simplify the util module, remove the third-party libraries iconv dependencies
* Rearrange the error code
ver 2.1.1 2015-02-05 calvin
* Stripping of third-party libraries fasterxml,fasterjson dependencies
ver 2.0.1 2015-02-04 calvin
* Fixed a bug
* WINDOWS Supported
ver 2.0.0 2015-02-01 calvin
* support concurrents in single process
ver 1.0.0 2014-12-12 calvin
* create
Loading...
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/donggo/simspider.git
[email protected]:donggo/simspider.git
donggo
simspider
simspider
master

搜索帮助