tlater
/
scrape-yuanyuan
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
scrape-yuanyuan/qinghai
History
Tristan Daniël Maat f7cf03d442
Add dumped qinghai articles 		2022-04-09 19:57:08 +01:00
..
2020-02-08_275.html Handle special case 275 2022-04-09 19:31:09 +01:00
README.md Add dumped qinghai articles 2022-04-09 19:57:08 +01:00
articles-qinghai.zip Add dumped qinghai articles 2022-04-09 19:57:08 +01:00
scrape-iframe.py Handle special case 275 2022-04-09 19:31:09 +01:00
scrape.py Work around other broken links 2022-04-09 19:30:41 +01:00

README.md

Qinghai scraping

Zip of full article dump: articles-qinghai.zip.

A few links don't exist anymore. They have the indexes 210, 453, 681, 703, 791, 871, 913, 914, 915 in links.csv.

There are a few small files again, mostly pdf links:

.rw-r--r-- 101 tlater  9 Apr 19:26 ./2016-09-28_923.txt
.rw-r--r-- 133 tlater  9 Apr 19:26 ./2016-09-28_924.txt
.rw-r--r-- 116 tlater  9 Apr 19:26 ./2016-09-28_925.txt
.rw-r--r-- 147 tlater  9 Apr 19:26 ./2016-09-28_926.txt
.rw-r--r-- 111 tlater  9 Apr 19:23 ./2017-03-16_838.txt
.rw-r--r--  36 tlater  9 Apr 19:20 ./2017-07-07_745.txt
.rw-r--r--  82 tlater  9 Apr 19:17 ./2017-08-14_723.txt
.rw-r--r-- 211 tlater  9 Apr 19:17 ./2017-09-12_704.txt
.rw-r--r--  97 tlater  9 Apr 19:14 ./2017-11-15_587.txt
.rw-r--r-- 156 tlater  9 Apr 19:13 ./2017-11-20_580.txt
.rw-r--r-- 283 tlater  9 Apr 19:13 ./2017-11-23_575.txt
.rw-r--r--  39 tlater  9 Apr 19:13 ./2017-12-29_566.txt
.rw-r--r--  39 tlater  9 Apr 19:13 ./2018-01-12_561.txt
.rw-r--r-- 165 tlater  9 Apr 19:12 ./2018-05-30_505.txt
.rw-r--r-- 145 tlater  9 Apr 19:12 ./2018-05-30_507.txt
.rw-r--r-- 391 tlater  9 Apr 19:11 ./2018-07-25_475.txt
.rw-r--r-- 158 tlater  9 Apr 19:11 ./2018-09-13_467.txt
.rw-r--r-- 204 tlater  9 Apr 19:04 ./2020-03-09_254.txt
.rw-r--r-- 124 tlater  9 Apr 19:04 ./2020-03-18_248.txt
.rw-r--r-- 228 tlater  9 Apr 19:04 ./2020-03-20_245.txt
.rw-r--r-- 186 tlater  9 Apr 19:03 ./2020-04-01_221.txt
.rw-r--r--  67 tlater  9 Apr 19:02 ./2020-04-21_208.txt
.rw-r--r-- 174 tlater  9 Apr 19:01 ./2020-04-30_194.txt
.rw-r--r-- 147 tlater  9 Apr 19:01 ./2020-05-08_186.txt
.rw-r--r-- 189 tlater  9 Apr 19:01 ./2020-05-12_182.txt
.rw-r--r--  82 tlater  9 Apr 19:01 ./2020-05-15_180.txt
.rw-r--r-- 119 tlater  9 Apr 19:00 ./2020-06-04_139.txt
.rw-r--r-- 201 tlater  9 Apr 19:00 ./2020-07-01_114.txt
.rw-r--r-- 113 tlater  9 Apr 18:59 ./2020-07-20_90.txt
.rw-r--r-- 115 tlater  9 Apr 18:59 ./2020-07-21_86.txt
.rw-r--r--  99 tlater  9 Apr 18:58 ./2020-08-27_36.txt
.rw-r--r--  99 tlater  9 Apr 18:58 ./2020-08-27_37.txt
.rw-r--r-- 130 tlater  9 Apr 18:58 ./2020-08-27_38.txt
.rw-r--r-- 130 tlater  9 Apr 18:58 ./2020-08-27_39.txt
.rw-r--r-- 190 tlater  9 Apr 18:58 ./2020-08-27_40.txt
.rw-r--r-- 190 tlater  9 Apr 18:58 ./2020-08-27_41.txt
.rw-r--r-- 184 tlater  9 Apr 18:58 ./2020-08-27_42.txt
.rw-r--r-- 184 tlater  9 Apr 18:58 ./2020-08-27_43.txt
.rw-r--r-- 127 tlater  9 Apr 18:58 ./2020-08-27_44.txt
.rw-r--r-- 127 tlater  9 Apr 18:58 ./2020-08-27_45.txt
.rw-r--r--  94 tlater  9 Apr 18:58 ./2020-08-27_46.txt
.rw-r--r--  94 tlater  9 Apr 18:58 ./2020-08-27_47.txt
.rw-r--r--  88 tlater  9 Apr 18:58 ./2020-09-12_20.txt
.rw-r--r-- 200 tlater  9 Apr 18:57 ./2020-09-21_11.txt