Pamyat Naroda indexes. 3,400,000+ records processed

Discussions on archives and similar issues. Hosted by John Calvin and Jeff Leach.
Mori
Member
Posts: 1301
Joined: 25 Oct 2014 11:04
Location: Europe

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Mori » 03 Apr 2019 08:54

fisenfender wrote:
02 Apr 2019 22:58
Pamyat Naroda is a unique site. I do not think that there are similar websites in the world with so many documents.
The Canadian site is even better. Also with a impressive mass of documents, not only Canadian proper but also everything they got cc-ed of from Americans and British.

Mori
Member
Posts: 1301
Joined: 25 Oct 2014 11:04
Location: Europe

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Mori » 03 Apr 2019 08:56

fisenfender wrote:
02 Apr 2019 22:58
But the site at any time may cease to exist and close, despite the financing of the state. There are a lot of things held up by enthusiasm.(...)
There are several cases downloaded from there, so if anyone needs or is interested, write me.
I believe several people already "scrapped" the site, ie made a systematic download of everything there is.

Is your proposal commercial?

User avatar
Jeff Leach
Host - Archive section
Posts: 1392
Joined: 19 Jan 2010 09:08
Location: Stockholm, Sweden

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Jeff Leach » 04 Apr 2019 07:35

Mori wrote:
03 Apr 2019 08:56
fisenfender wrote:
02 Apr 2019 22:58
But the site at any time may cease to exist and close, despite the financing of the state. There are a lot of things held up by enthusiasm.(...)
There are several cases downloaded from there, so if anyone needs or is interested, write me.
I believe several people already "scrapped" the site, ie made a systematic download of everything there is.
I doubt that there are 'complete' copies in private hands. They might be 80-90% complete. New material is being added all the time and without a release schedual, you would need to scan the site regularly. If they are using dynamic links, then you would need to compare the actual files instead of the link addresses. Someone, I know was trying to download all the files and they had around 25 TB of material.

My greatest worry, is that the site will close unexpectedly. I try and keep an updated catalogue of the files I need for my research but it is still a pain just checking on the 10 - 20,000 pages of documents that interest me.

User avatar
AMVAS
Member
Posts: 528
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 04 Apr 2019 18:41

I doubt that there are 'complete' copies in private hands. They might be 80-90% complete.
Even less. About 50% of operative records cost me 28 Tb. Hardly too much personalities have similar HDD capacities.
New material is being added all the time and without a release schedual, you would need to scan the site regularly.
It's enough to scan annually. No need to do this every month.
The problem is they have changed their search engine and one need to make a special research to learn how to get access to it.
If they are using dynamic links, then you would need to compare the actual files instead of the link addresses. Someone, I know was trying to download all the files and they had around 25 TB of material.
Probably it was me ))
My greatest worry, is that the site will close unexpectedly. I try and keep an updated catalogue of the files I need for my research but it is still a pain just checking on the 10 - 20,000 pages of documents that interest me.
Moreover they quite often delete some files from access without any informatiopm about this.

Mihai Pica
Member
Posts: 41
Joined: 21 Apr 2017 12:25
Location: Bucharest, Romania

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Mihai Pica » 16 Dec 2020 09:15

Sorry for the bump, does anyone know if there is any way to check only for the newer files ( if any ? )

User avatar
AMVAS
Member
Posts: 528
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 16 Dec 2020 10:00

Mihai Pica wrote:
16 Dec 2020 09:15
Sorry for the bump, does anyone know if there is any way to check only for the newer files ( if any ? )
This is one of the main puzzles for this site :D

Mihai Pica
Member
Posts: 41
Joined: 21 Apr 2017 12:25
Location: Bucharest, Romania

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Mihai Pica » 16 Dec 2020 11:52

Well..what can I say..will keep playing the game :D

User avatar
AMVAS
Member
Posts: 528
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 16 Dec 2020 13:29

Mihai Pica wrote:
16 Dec 2020 11:52
Well..what can I say..will keep playing the game :D
I tried to index their records. But that %^$$% made dynamic address for search engine! I never met such a level of paranoia :D So I have only index for 2016 year

User avatar
Der Alte Fritz
Member
Posts: 2076
Joined: 13 Dec 2007 21:43
Location: Kent United Kingdom

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Der Alte Fritz » 18 Dec 2020 07:14

Site recently lost the download button and image files can be downloaded directly from screen with a left click of mouse button so an index scan is now possible.

User avatar
AMVAS
Member
Posts: 528
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 18 Dec 2020 09:59

Der Alte Fritz wrote:
18 Dec 2020 07:14
Site recently lost the download button and image files can be downloaded directly from screen with a left click of mouse button so an index scan is now possible.
before loadings they must be found first.

Lefty2000
Member
Posts: 4
Joined: 25 Mar 2015 15:10
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Lefty2000 » 29 Dec 2020 07:22

Mihai Pica wrote:
16 Dec 2020 09:15
Sorry for the bump, does anyone know if there is any way to check only for the newer files ( if any ? )
Newer files have higher IDs. I've recently updated indexes, you can download it here.

You can also sort by ID on https://vnr.github.io/pamyat-naroda-search/ (unfortunately that site might stop functioning at any time if pamyat-naroda changes the algorithm to access its database once again).

Mihai Pica
Member
Posts: 41
Joined: 21 Apr 2017 12:25
Location: Bucharest, Romania

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Mihai Pica » 29 Dec 2020 09:47

Lefty2000 wrote:
29 Dec 2020 07:22

Newer files have higher IDs. I've recently updated indexes, you can download it here.

You can also sort by ID on https://vnr.github.io/pamyat-naroda-search/ (unfortunately that site might stop functioning at any time if pamyat-naroda changes the algorithm to access its database once again).
Lefty thanks so much ! This seems super useful, will try right away !

PS: I seem to find very few aviation units, are these excluded from you search engine ?

Lefty2000
Member
Posts: 4
Joined: 25 Mar 2015 15:10
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Lefty2000 » 29 Dec 2020 18:06

Mihai Pica wrote:
29 Dec 2020 09:47
PS: I seem to find very few aviation units, are these excluded from you search engine ?
Are you sure that some aviation units documents exist on https://pamyat-naroda.ru/documents/ but missed on github? Could you provide me with an example?

Mihai Pica
Member
Posts: 41
Joined: 21 Apr 2017 12:25
Location: Bucharest, Romania

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Mihai Pica » 30 Dec 2020 11:29

You are right, everything is in order, my bad russian skills are to blame, some documents come up when I search for the "Military Unit", but forgot about those where the unit is in the "Document Author" field.

Thanks !

Return to “Archives”