Offline URL Database 1st April 2022 changelog

From URL Classification
Revision as of 13:42, 11 April 2022 by Barak (talk | contribs) (Created page with "= Category change = * Added Prevent crime or violence category * Added Collaboration software category = Database change = * Removed duplicate domains that had both regular...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Category change

  • Added Prevent crime or violence category
  • Added Collaboration software category

Database change

  • Removed duplicate domains that had both regular host name and www preceding host name, for example www.cnn.com and cnn.com the www.cnn.com entry was removed.

Domain count and statistics

ID Name Domain count
1 Abortion and pregnancy 105,078
2 Ads 77,879
3 Adult 467,097
4 Adult dating 19,240
5 Alcohol 423,652
6 Astrology 65,239
7 Blogs 3,277,602
8 Books or comics 517,662
9 Business and services 6,421,817
10 CDN 1,021,343
11 Chats 107,690
12 Classifieds 121,063
13 Clothing and fashion 1,473,529
14 Cooking and food 1,894,674
15 Dating 94,483
16 Drugs 74,754
17 Education 1,258,859
18 Empty site 2,016,005
19 Entertainment 765,886
20 Error site can't be accessed 4,730,071
21 DNS error site can't be resolved 41,049,739
22 File hosting 66,480
23 Finance (Banks_Real estate_Insurance) 515,011
24 Gambling 370,042
25 Computer games 538,844
26 General (safe) 10,752
27 Government 141,131
28 Hacking 10,648
29 Hate 21,082
30 Hobbies 399,182
31 Home improvement 498,139
32 Humor 31,030
33 Hunting and fishing 115,222
34 Image and video hosting 99,026
35 Jobs 865,539
36 Kids safe 221
37 Legal 463,467
38 Life style and beauty 767,610
39 Link farm (no real content) 5,344,739
40 Local information 73,875
41 Medical 2,751,531
42 Military 31,871
43 Mixed adult 1,014,425
44 Mobile 290,841
45 Models 57,879
46 Music 1,156,106
47 News 1,664,222
49 Not child safe 26,786
50 Online drugs 36,200
51 Online marketing 581,814
52 Online video or audio 323,686
53 Pets and animals 428,721
54 Phishing 73
55 Politics 383,185
56 Porn 824,603
57 Internet portals 337,232
58 Profanity 30,826
59 Proxy 15,106
60 Religion 583,684
62 Science 338,604
63 Search engines 65,867
64 Sects 13,769
65 Semi nude and pin up 26,723
66 Sex education 68,750
67 Shopping 2,706,864
68 Social networks 945,528
69 Spam or scam 3,902
70 Sports 1,354,268
71 Suicide 295
72 Technology and computers 1,018,324
73 Site under construction or not available 5,683,463
74 General - not porn 9,706,113
75 URL shortener 5,536
76 Vacation and travel 1,873,001
77 Vehicles 1,296,981
78 Violence 32,613
79 Virus or Spyware 1,759
80 Warez 31,570
81 Weapons 148,819
82 Web hosting 900,607
83 Web mail 77,798
84 Reference 215,267
85 Community and society 193,133
86 Software repository 29,751
87 Philosophy and psychology 252,119
88 History 286,278
89 Nonsense and time waste 76,599
90 Local IP 272
91 Swimsuits and underwear 110,311
92 Arts 740,533
94 Login page 120,335
95 Surveys and polls 33,171
96 Tobacco 56,968
97 ISP 18,669
98 Tracker 1,467,240
99 Bypass protection 1,731
100 Search block 51
101 Image search block 142,585
102 Desktop themes 9,547
103 Depress domain 73,867
104 Depress page 8,569
105 Parenting and family 77,247
106 School cheats 4,522
107 Celebrity gossip 127,198
108 How to 284,979
109 Illegal 2,918
110 Recreation 212,827
111 Bookmarks 1,244
112 Money spam 2,194
113 Directories 5,027
114 Site_Graphic design 470,559
115 Weather 83,251
116 Knowledge 62,039
117 Kids entertainment 16,007
118 Nature 110,131
119 Diet 128,776
120 Eating disorders 5,338
121 Self help 100,465
122 Relationships 20,646
123 Paranormal 39,014
124 Images CDN 880
125 Images CDN Unknown 57,413
126 LGBT 14,792
127 Marketing or ads 43,637
128 Magazines 154,126
129 Forums 129,850
130 User generated content 48,249
131 Tele Communications 54,921
132 Gender specific 2,428
134 Crime 56,053
135 Person 58,133
136 Grief 2,132
137 Retirement 53,735
138 Other drugs 56,542
139 Pharmacy 42,909
140 Songs download 16,040
141 Lookup 38,034
142 Movies download 109,599
143 Consumer reviews 18,803
144 API 63,048
145 Legal movies 494
146 Legal songs 505
147 Bitcoin 60,604
148 Victim help 2,634
149 Rewards 8,396
150 Photography 406,181
151 Toys 125,126
152 Restaurant 388,488
153 Fitness 285,171
154 Contact sport 130,909
155 Dancing 55,098
156 Software 431,453
157 Programming 247,651
158 Registry cleaner 275
159 Online security 120,827
160 Real estate 787,166
161 Loans 134,746
162 Forex 28,123
163 Banks 256,714
164 Insurance 293,870
165 SEO 88,275
166 PPC 7,968
167 Supplements 46,505
168 URL Classification 470
170 Tatoo 31,265
172 Tactical equipment 4,212
173 Alternative medicine 239,358
174 AI 16,626
175 Comics 22,950
176 Manga 6,228
177 Captcha 70,478
178 Racism education 1,239
179 Payment 9,509
180 Brokerage 19,805
181 SAAS 7,657
182 Anime 3,187
183 Cartoon 1,001
184 Grownups 18
185 Stream converter 2,359
186 File converter 2,110
187 TV Personality 890
188 Radio Personality 559
189 Youtube Personality 7,690
190 Prevent crime or violence 1,941
191 Collaboration software 1,249

Total domains: 124,126,029

Total classified domains: 78,346,219 (Total domains excluding DNS resolve errors and other errors)

Total unique domains: 112,763,103

Total unique classified domains: 67,021,911 (Total unique domains excluding DNS resolve errors and other errors)

TLD Distribution (all domains)

# TLD Domain count
1 com 56,937,771
2 net 6,280,370
3 de 4,217,363
4 org 4,179,135
5 uk 3,440,045
6 ru 2,273,849
7 cn 1,811,712
8 info 1,709,631
9 nl 1,368,011
10 fr 1,168,873
11 pl 1,110,499
12 it 1,095,139
13 jp 1,091,608
14 au 1,045,458
15 br 1,006,333
16 tk 906,159
17 tw 702,553
18 ca 700,481
19 eu 626,255
20 top 575,890
21 in 566,421
22 co 556,697
23 biz 537,412
24 ch 537,009
25 us 519,408
26 es 479,937
27 cz 463,141
28 se 424,982
29 be 394,749
30 at 378,711
31 ga 355,056
32 cf 336,760
33 ir 332,337
34 dk 318,193
35 za 305,937
36 hu 290,687
37 club 281,117
38 ml 274,294
39 nz 266,077
40 gq 265,138
41 ua 260,150
42 me 253,144
43 ro 251,826
44 home 251,468
45 tr 248,146
46 io 244,109
47 cc 243,355
48 mx 213,723
49 ar 211,246
50 icu 197,622
51 gr 195,498
52 kr 191,257
53 online 190,188
54 xyz 186,055
55 no 181,606
56 il 180,329
57 loan 174,195
58 live 170,288
59 site 170,284
60 fi 167,151
61 pw 163,801
62 vn 153,848
63 tv 145,157
64 sk 144,519
65 cl 143,545
66 edu 137,125
67 id 120,602
68 pt 108,845
69 xn--p1ai 104,775
70 pro 103,057
71 ie 99,963
72 ws 93,449
73 nu 78,514
74 lt 75,725
75 vip 73,137
76 kz 70,970
77 my 68,832
78 gdn 68,478
79 space 68,318
80 shop 66,716
81 by 65,496
82 su 61,933
83 win 61,702
84 sg 60,639
85 hk 59,952
86 asia 59,896
87 bid 59,871
88 si 57,532
89 website 57,414
90 to 53,685
91 hr 52,758
92 ee 52,082
93 mobi 50,865
94 rs 50,018
95 tech 48,904
96 domain 46,213
97 lv 45,290
98 gov 44,823
99 la 42,100
100 All other 5,329,555

TLD Distribution (excluding DNS resolve errors and other errors)

# TLD Domain count
1 com 36,894,515
2 net 3,717,334
3 de 2,892,826
4 org 2,513,637
5 uk 2,464,481
6 ru 1,347,351
7 nl 1,027,297
8 fr 785,278
9 jp 780,541
10 it 773,084
11 pl 725,132
12 au 711,980
13 br 621,858
14 cn 601,986
15 ca 525,483
16 info 442,845
17 ch 414,413
18 cz 362,255
19 eu 358,656
20 es 327,380
21 in 307,917
22 se 306,133
23 co 294,474
24 be 289,601
25 at 284,865
26 dk 239,370
27 hu 218,700
28 za 213,995
29 us 202,690
30 biz 196,573
31 ua 177,315
32 io 173,507
33 nz 170,732
34 ir 162,539
35 tr 159,230
36 ro 156,541
37 ga 154,529
38 cf 147,568
39 no 143,664
40 gr 138,896
41 fi 135,448
42 ml 134,713
43 mx 132,913
44 kr 125,520
45 me 125,055
46 gq 124,020
47 ar 123,700
48 il 113,440
49 sk 110,107
50 edu 106,397
51 tw 105,918
52 vn 100,475
53 cl 85,974
54 cc 84,547
55 pt 81,545
56 ie 76,703
57 tv 74,187
58 id 67,944
59 xn--p1ai 63,133
60 ws 62,898
61 lt 59,331
62 online 55,238
63 my 46,812
64 club 46,524
65 by 45,636
66 site 44,648
67 si 42,028
68 ee 41,830
69 xyz 40,416
70 pro 39,915
71 sg 39,605
72 top 38,439
73 hk 38,120
74 hr 37,993
75 rs 36,039
76 nu 35,888
77 tk 35,863
78 lv 33,202
79 kz 31,101
80 su 28,085
81 ph 27,607
82 th 25,955
83 bg 25,684
84 cat 25,088
85 app 24,592
86 gov 24,500
87 shop 22,791
88 blog 22,353
89 ae 21,234
90 pe 20,826
91 pk 20,687
92 live 20,006
93 mobi 18,479
94 is 17,433
95 lu 16,201
96 space 14,824
97 asia 14,560
98 to 14,548
99 tech 13,416
100 All other 1,352,602