Here in the United Kingdom, today is "census day". The 32-page booklet has been in the house for a couple of weeks. We have glanced at it, planning how we might answer the questions closer to the time. I noticed that we could answer it online, and that is what I decided to do. What to do with 32 pages of unused but printed paper in these days of recycling was an instruction omitted from the census form.
The standard form allowed for a household of six people--a perfect fit for our neighbours who have four children, but somewhat excessive for us--a married couple with one son, still single, at home. One of the questions was to relate each of the people in the house on census day to each of the others. Oh, how I wish the census designers of 1861 had thought up such a question. It would have solved so many presumptions I have made.
A couple of censuses ago the questions on birthplace and ethnical background were very few. Political Correctness was the way of the moment. Not any more. This year the choices of ethnicity cover a whole page for each member of the household. It was great to be different, to be able to tick "Other" and explain "Canadian" or "Canada" three times on my part of the form. And then put the icing on the cake by stating that I had lived in Britain since August 1965.
As per usual there were questions about the size of our home, whether it was a house or a flat, how many rooms it had, and how it was heated. We were also asked how many cars the household had. It wasn't until the three of us sat down to dinner this evening that we realized we had not been asked about computer usage. How many computers in the house? Do you have broadband? Do you have HD television (an abbreviation I always translate as heavy duty, but turns out to be high density)? Not one of these questions was included.
Ten years ago, at the last census, offering the facility to fill in the census over the internet would have been a fruitless thing to do. Now, it is simple and straightforward for many households all over the country. It will be interesting to see what percentage of the population answered our questionnaires that way.
Sunday, 27 March 2011
Monday, 14 February 2011
Brown's Directory Now a Database
It was January 11 when I wrote that I was attempting to make a database from an OCR scan of Brown’s Directory of Toronto dated 1861-62. On Friday (11 February), I had a listing of the complete street directory of approximately 8500 entries. OCR is workable on 19th century printing after all, but find-and-replace was far more useful than the automated spell checker that the software makers expect users to employ.
I have also experimented with using OCR on the “names directory” in Brown . Having got as far as the D’s, I have decided that it is not worth the effort. In the “street directory” people are arranged by their addresses and the street names only come up as headers; while, in the names directory, there is an address on every line. OCR can find no uniformity in these street names, so there is no time saved over copy-typing. It also appears that entries in the alphabetical list are one per house, no more than in the street directory. Mitchell’s Directory of three years later included most employed people whether they were head of household or not. There is not point on duplicating Brown’s Directory by transcibing a second lot of the same information.
The work of matching the census to my new directory database had now begun. The first ward I attacked was St Andrew’s which included many forms for commercial premises and where street names were omitted for the first division (Yonge Street west to York, Queen West to Adelaide). I had added as many clues from these forms as I could to my transcription including all notes stating “Personal details at place of residence”, signed by the proprietor and, maybe, giving his home address. Two of the first three forms with nameless proprietors have now been matched to actual people. For instance, Mr Leask’s handwriting was very hard to read and I had not been able to recognize the street he lived on, although the house number was 174. A quick inspection of my sorted digitized database found him at 174 Gerrard Street East, as well as the nature of his business downtown.
This kind of discovery challenged me to continue the census matching process. I have now finished the letter A for five wards. Two more city wards and the suburbs of Yorkville and East York to go. I hope temptations won’t lead me astray before the end of the alphabet. Certainly, if Brown’s Directory had come into my hands earlier, Caverhill and Mitchell would never have been done.
I have also experimented with using OCR on the “names directory” in Brown . Having got as far as the D’s, I have decided that it is not worth the effort. In the “street directory” people are arranged by their addresses and the street names only come up as headers; while, in the names directory, there is an address on every line. OCR can find no uniformity in these street names, so there is no time saved over copy-typing. It also appears that entries in the alphabetical list are one per house, no more than in the street directory. Mitchell’s Directory of three years later included most employed people whether they were head of household or not. There is not point on duplicating Brown’s Directory by transcibing a second lot of the same information.
The work of matching the census to my new directory database had now begun. The first ward I attacked was St Andrew’s which included many forms for commercial premises and where street names were omitted for the first division (Yonge Street west to York, Queen West to Adelaide). I had added as many clues from these forms as I could to my transcription including all notes stating “Personal details at place of residence”, signed by the proprietor and, maybe, giving his home address. Two of the first three forms with nameless proprietors have now been matched to actual people. For instance, Mr Leask’s handwriting was very hard to read and I had not been able to recognize the street he lived on, although the house number was 174. A quick inspection of my sorted digitized database found him at 174 Gerrard Street East, as well as the nature of his business downtown.
This kind of discovery challenged me to continue the census matching process. I have now finished the letter A for five wards. Two more city wards and the suburbs of Yorkville and East York to go. I hope temptations won’t lead me astray before the end of the alphabet. Certainly, if Brown’s Directory had come into my hands earlier, Caverhill and Mitchell would never have been done.
Tuesday, 1 February 2011
Working with OCR software
I was trying so hard not to digitize Brown’s Toronto Directory. I thought it would take up too much time and I really was trying to complete the linking of the other two directories to the people who had the fortune or misfortune to be represented amongst the inhabitants of "Institutions" in the 1861 census. About a month ago I succumbed. Since then I have been ensconced in yet another data collection dealing with Toronto in 1861.
First, was the compiler, W R Brown, part of the Brown family of The Globe, or was he somebody else? Originally I assumed he was part of the newspaper family, but I am beginning to wonder if my assumption was correct. Living at Idlewold in Rosedale would have meant a long trip to the office everyday. Either his knowledge of downtown Toronto or a great mixup of entries just before typesetting led to a very noticeable error on Front Street West. The south side of the street suddenly jumped from Bay to Simcoe omitting Jacques & Hays’ cabinet works and Union Station. Numbered premises began again after Simcoe with even numbers which should have been on the north side. All the entries actually existed, but the order spoiled the meaning of the word “index”.
When I first discussed indexing city directories back in November I said that optical character recoginition (OCR) was not the way to go with 19th century printed pages. A couple of weeks ago I decided to see if another attempt with some software I bought years ago would be more successful.
The pages of the directory are filed individually in my computer and each page has two columns of entries. I open each directory page in my photo-finishing/drawing program, Paint.net, where I straighten the page and crop each of the columns to individual images or files. Then I take the separate columns to the OCR program and re-crop to get rid of extraneous lines and blobs of dirt on the image. The image is then “read” by the software. The result is not a pretty sight. To get one line completely correct on a page of 60 entries is unusual. The software is exceedingly poor at distinguishing numbers like “3” and “8” and “6” and “0”. The letters “v” and “y” are almost always misinterpreted. One gets used to these little vagaries.
A new discovery for me was that the OCR software permits find-and-replace corrections. I now start off work on each page by getting rid of unwanted double and triple spaces and remove all periods or full stops—things that aren’t wanted in my database style. Other start-up operations include altering the font to a uniform 20-point Times New Roman and typing a title right on the page—usually the book page number and the street it contains. Correcting a column takes about 20 minutes. Each column is saved to an individual spreadsheet file.
Once I have a collection of 10 to 20 completed spreadsheets I take a break from OCR work and combine the spreadsheets into one large one where I arrange the data into a series of columns which will ultimately be moved to a database.
The whole process probably takes as long as copy-typing the whole book, but because there is such a variety of tasks involved, the boredom factor is reduced. Into the bargain, my aging body is subject to a lot of aches and pains, particularly in the neck and shoulders. Continuous copy-typing would send me into complete lock-down. The other advantage, a personal one, is that, once I have gone through both the OCR and spreadsheet steps, I have a much better knowledge of the street I have been looking at and the people who lived there.
First, was the compiler, W R Brown, part of the Brown family of The Globe, or was he somebody else? Originally I assumed he was part of the newspaper family, but I am beginning to wonder if my assumption was correct. Living at Idlewold in Rosedale would have meant a long trip to the office everyday. Either his knowledge of downtown Toronto or a great mixup of entries just before typesetting led to a very noticeable error on Front Street West. The south side of the street suddenly jumped from Bay to Simcoe omitting Jacques & Hays’ cabinet works and Union Station. Numbered premises began again after Simcoe with even numbers which should have been on the north side. All the entries actually existed, but the order spoiled the meaning of the word “index”.
When I first discussed indexing city directories back in November I said that optical character recoginition (OCR) was not the way to go with 19th century printed pages. A couple of weeks ago I decided to see if another attempt with some software I bought years ago would be more successful.
The pages of the directory are filed individually in my computer and each page has two columns of entries. I open each directory page in my photo-finishing/drawing program, Paint.net, where I straighten the page and crop each of the columns to individual images or files. Then I take the separate columns to the OCR program and re-crop to get rid of extraneous lines and blobs of dirt on the image. The image is then “read” by the software. The result is not a pretty sight. To get one line completely correct on a page of 60 entries is unusual. The software is exceedingly poor at distinguishing numbers like “3” and “8” and “6” and “0”. The letters “v” and “y” are almost always misinterpreted. One gets used to these little vagaries.
A new discovery for me was that the OCR software permits find-and-replace corrections. I now start off work on each page by getting rid of unwanted double and triple spaces and remove all periods or full stops—things that aren’t wanted in my database style. Other start-up operations include altering the font to a uniform 20-point Times New Roman and typing a title right on the page—usually the book page number and the street it contains. Correcting a column takes about 20 minutes. Each column is saved to an individual spreadsheet file.
Once I have a collection of 10 to 20 completed spreadsheets I take a break from OCR work and combine the spreadsheets into one large one where I arrange the data into a series of columns which will ultimately be moved to a database.
The whole process probably takes as long as copy-typing the whole book, but because there is such a variety of tasks involved, the boredom factor is reduced. Into the bargain, my aging body is subject to a lot of aches and pains, particularly in the neck and shoulders. Continuous copy-typing would send me into complete lock-down. The other advantage, a personal one, is that, once I have gone through both the OCR and spreadsheet steps, I have a much better knowledge of the street I have been looking at and the people who lived there.
Tuesday, 11 January 2011
Another Directory Comes to Light
Sometime in the autumn, Ancestry put a number of city directories for Canadian towns and cities online. Included amongst these is Brown's Toronto Directory for 1861-2. It proves to be a very worthwhile addition to the collection of listings of people in the Toronto area in the early 1860s period, and the closest one in time to the census.
The directories I have seen before were published by men who were not permanent Toronto residents, and they may have employed agents to do the job. (Mitchell's compiler got his own name in the directory.) It was a temptation to cut corners, or perhaps I should say numbers of pages, to reduce the cost of publication for one of a series of books covering different cities across the provinces. But George Brown was producing a directory for the city he lived in--he knew its component parts and they included suburbia. His street directory includes the streets of Yorkville and the area east of the Don River between the Lakefront and Todmorden.
We know of George Brown of The Globe, but The Globe was very much a family firm. In 1861, although over 40, George Brown was still living with his parents. His father, Peter, who had started the Toronto newspaper business with his son, was still working in the office. George's brother, Gordon, was actually editor of The Globe. W R Brown was the publisher of the Toronto Directory. Perhaps because he lived at Idlewold, a house on the Yorkville-Rosedale border, he thought it worthwhile to include the areas outside the formal city limits in his publication.
Because of its place in the alphabet, Bloor Street was the first of the Yorkville streets I have found. The south side of Bloor Street was in Toronto, the north side between the top of the Don River ravine and what we now call Avenue Road was in Yorkville. Further west, Bloor Street West continued into York Township Electoral District 3. Mitchell omitted the north side of Bloor Street from his street directory of 1864, but Brown includes it in 1862. Many of the people in the Yorkville section are well-known; those further west, lesser so. But, at least, they are there.
I am looking forward to discovering more gems from Brown's Toronto Directory. Already it has told me more than I knew before.
The directories I have seen before were published by men who were not permanent Toronto residents, and they may have employed agents to do the job. (Mitchell's compiler got his own name in the directory.) It was a temptation to cut corners, or perhaps I should say numbers of pages, to reduce the cost of publication for one of a series of books covering different cities across the provinces. But George Brown was producing a directory for the city he lived in--he knew its component parts and they included suburbia. His street directory includes the streets of Yorkville and the area east of the Don River between the Lakefront and Todmorden.
We know of George Brown of The Globe, but The Globe was very much a family firm. In 1861, although over 40, George Brown was still living with his parents. His father, Peter, who had started the Toronto newspaper business with his son, was still working in the office. George's brother, Gordon, was actually editor of The Globe. W R Brown was the publisher of the Toronto Directory. Perhaps because he lived at Idlewold, a house on the Yorkville-Rosedale border, he thought it worthwhile to include the areas outside the formal city limits in his publication.
Because of its place in the alphabet, Bloor Street was the first of the Yorkville streets I have found. The south side of Bloor Street was in Toronto, the north side between the top of the Don River ravine and what we now call Avenue Road was in Yorkville. Further west, Bloor Street West continued into York Township Electoral District 3. Mitchell omitted the north side of Bloor Street from his street directory of 1864, but Brown includes it in 1862. Many of the people in the Yorkville section are well-known; those further west, lesser so. But, at least, they are there.
I am looking forward to discovering more gems from Brown's Toronto Directory. Already it has told me more than I knew before.
Wednesday, 8 December 2010
City Directories and the Census
Part of my 1861 Toronto census project has involved matching people in the census to entries in city directories of the time, namely that of Caverhill in 1859-60 and of Mitchell in 1863-64. This has proved very interesting. There is evidence of a lot of movement around the city, as well as of people coming into the city and disappearing from it. From the census we learn a person’s age and his birthplace. From the directories we learn the kind of business he may be running or employed in. We may discover that a woman has been widowed during the five-year period. And I find hints that I may have misinterpreted a surname when transcribing the census. It has certainly been a worthwhile exercise.
In order to use the directories efficiently I have had to digitize them. Digitizing is a word we have heard in connection with computers over the past ten years, but do we know what it means? Taken simply, it is the process of copy-typing the directory entries onto a computer, and, at the same time, breaking down each line into its component parts of surname, given name, occupation and address. If you have transcribed a census or a list of burials in a Family Search project, you have digitized the manuscript information on the original forms.
Why should we have to do this for printed books? Can’t books be scanned and read by optical character recognition? Well, many more recently printed ones can, but 19th-century type does not lend itself to the procedure. The letters are just not sharp enough for a camera to identify each as one of the twenty-six used in our English alphabet. I have tried OCR programs on pages of old data and come very close to turning the air blue trying to make sense of the software’s interpretation. It might take a long time, but old fashioned copy-typing is the better plan of attack.
Since I was transcribing the directories myself I was able my own choices for headings. My initial reason for looking at the directories was that some enumerators had omitted addresses from the census forms. I wanted to know where these mystery areas actually were. For my purposes it was best to separate the directory addresses into streets and numbers and add in the city ward where the streets were located. From this I could make up a list of inhabitants, street by street, and house by house.
Both directories had a number of foibles. Caverhill’s inability to keep his records in strict alphabetical order was frustrating. I am suspicious that a pile of entries at the end of the alphabet went astray before the directory was printed. Initial letters W and Y are very poorly represented. He also had a bad habit of omitting the “East” or the “West” from streets that crossed Yonge. An address like 394 Queen could either be around Denison Avenue or between Sumach and River Streets, a very long way apart.
Mitchell’s Directory of 1864 had two parts and in one the households were arranged by streets. I started my database using this, but then found it only contained house owners or the heads of the tenant families. His alphabetical one contained not just heads but other people in the house: tenants, boarders, mothers-in-law, sons (and daughters) who were working or who were students in law or at university. Sometimes the addresses in the street directory differed from those in the alphabetical directory indicating that the family had moved house sometime during the compilation process. The alphabetical listing usually contained the later address. Mitchell had obtained staff lists provided by large employers. Unfortunately, many people’s addresses were missing. But the directory had an extra advantage: the entries for most widowed women contained the phrase “widow of John” or “widow of William”. How useful to the genealogist.
We ought not to complain about the imperfections in a mid 19th century city directory. Picture how it must have been developed—without electricity, typewriters, and all the other paraphernalia that we have depended on for the last hundred years. Until the directory was finally typeset, all the listings must have been done with pen and paper. It would have been easy to misread a colleague’s handwriting or even one’s own on a cloudy day in a dusty office. Omitting a letter or a cross on a “t” can make a lot of difference. Were entries made on foolscap pieces of paper or on slips that fitted in a shoebox? (Given the number of shoemakers in Toronto at the time, they must have had shoeboxes.) I hope it was the latter. Corrections are much easier to make that way. However, the typesetters might have insisted on the listings being on sheets of papers—and the rewriting would be another step in the procedure where errors could creep in.
Today I have three “digitized” databases illustrating Toronto’s population over a five-year period at the beginning of the 1860s. I hope it will prove to be a more useful series of documents than any one of the three taken individually.
In order to use the directories efficiently I have had to digitize them. Digitizing is a word we have heard in connection with computers over the past ten years, but do we know what it means? Taken simply, it is the process of copy-typing the directory entries onto a computer, and, at the same time, breaking down each line into its component parts of surname, given name, occupation and address. If you have transcribed a census or a list of burials in a Family Search project, you have digitized the manuscript information on the original forms.
Why should we have to do this for printed books? Can’t books be scanned and read by optical character recognition? Well, many more recently printed ones can, but 19th-century type does not lend itself to the procedure. The letters are just not sharp enough for a camera to identify each as one of the twenty-six used in our English alphabet. I have tried OCR programs on pages of old data and come very close to turning the air blue trying to make sense of the software’s interpretation. It might take a long time, but old fashioned copy-typing is the better plan of attack.
Since I was transcribing the directories myself I was able my own choices for headings. My initial reason for looking at the directories was that some enumerators had omitted addresses from the census forms. I wanted to know where these mystery areas actually were. For my purposes it was best to separate the directory addresses into streets and numbers and add in the city ward where the streets were located. From this I could make up a list of inhabitants, street by street, and house by house.
Both directories had a number of foibles. Caverhill’s inability to keep his records in strict alphabetical order was frustrating. I am suspicious that a pile of entries at the end of the alphabet went astray before the directory was printed. Initial letters W and Y are very poorly represented. He also had a bad habit of omitting the “East” or the “West” from streets that crossed Yonge. An address like 394 Queen could either be around Denison Avenue or between Sumach and River Streets, a very long way apart.
Mitchell’s Directory of 1864 had two parts and in one the households were arranged by streets. I started my database using this, but then found it only contained house owners or the heads of the tenant families. His alphabetical one contained not just heads but other people in the house: tenants, boarders, mothers-in-law, sons (and daughters) who were working or who were students in law or at university. Sometimes the addresses in the street directory differed from those in the alphabetical directory indicating that the family had moved house sometime during the compilation process. The alphabetical listing usually contained the later address. Mitchell had obtained staff lists provided by large employers. Unfortunately, many people’s addresses were missing. But the directory had an extra advantage: the entries for most widowed women contained the phrase “widow of John” or “widow of William”. How useful to the genealogist.
We ought not to complain about the imperfections in a mid 19th century city directory. Picture how it must have been developed—without electricity, typewriters, and all the other paraphernalia that we have depended on for the last hundred years. Until the directory was finally typeset, all the listings must have been done with pen and paper. It would have been easy to misread a colleague’s handwriting or even one’s own on a cloudy day in a dusty office. Omitting a letter or a cross on a “t” can make a lot of difference. Were entries made on foolscap pieces of paper or on slips that fitted in a shoebox? (Given the number of shoemakers in Toronto at the time, they must have had shoeboxes.) I hope it was the latter. Corrections are much easier to make that way. However, the typesetters might have insisted on the listings being on sheets of papers—and the rewriting would be another step in the procedure where errors could creep in.
Today I have three “digitized” databases illustrating Toronto’s population over a five-year period at the beginning of the 1860s. I hope it will prove to be a more useful series of documents than any one of the three taken individually.
Wednesday, 24 November 2010
Exploring Mitchell's Directory
The end of a stage is in sight. This morning I have reached the letter "W" in matching the adult inhabitants of York Township to Mitchell's Directory of Toronto, published in 1864. Matching the inhabitants of Toronto itself to the city directory was completed some months ago.
Maybe it is getting to the end that sent me into hysterics about this entry found on page 180 of the directory:
Henry WHITE, explorer; 22 Nassau Street, St Patrick's
My first thought was, "Did he find it?" with the mind boggling about what there might be to look for on Nassau Street. But I've decided that "explorer" is a typo.
The census gives a much more sensible entry:
Henry WHITE, surveyor, 39, m, 5 children, Nassau Street, St Patrick's
It just shows that us census transcribers don't make all the mistakes.
Maybe it is getting to the end that sent me into hysterics about this entry found on page 180 of the directory:
Henry WHITE, explorer; 22 Nassau Street, St Patrick's
My first thought was, "Did he find it?" with the mind boggling about what there might be to look for on Nassau Street. But I've decided that "explorer" is a typo.
The census gives a much more sensible entry:
Henry WHITE, surveyor, 39, m, 5 children, Nassau Street, St Patrick's
It just shows that us census transcribers don't make all the mistakes.
Friday, 5 November 2010
Was Yonge Street Paved with Gold?
I have just found another family who must have improved their lot by moving.
In the 1861 census Ambrose Kent was a bricklayer in his early forties with two sons at home--Ambrose, 17, and Benjamin, 15. A third young man living with the family, but probably not a son, was Joseph who was working as a bricklayer. No occupations were given for Ambrose jr and Benjamin. They lived in Division Five of the Township which stretches northwest from Dufferin and Eglinton to the Humber River in the vicinity of Weston.
In Mitchell's Directory, compiled in 1863, Ambrose Kent was a grocer at 288 Yonge Street, i.e., in the middle of today's Eaton Centre. Ambrose jr and Benjamin were with him--Ambrose jr still didn't have an occupation, but Benjamin was now a watchmaker. John Kent (not found in the census) appeared next door at 286-1/2. There is a Joseph Kent on Walton Street (north of Gerrard but easy walking distance from Yonge and Queen). No other people of the surname Kent appear in the Directory.
There were two other Kent households in York Township, once again in Division Five. William, age 28, was another bricklayer, and F G Kent, described as a gentleman, age 47, with a wife nine years older than himself. Was this couple the source of capital for Ambrose's move?
In the 1861 census Ambrose Kent was a bricklayer in his early forties with two sons at home--Ambrose, 17, and Benjamin, 15. A third young man living with the family, but probably not a son, was Joseph who was working as a bricklayer. No occupations were given for Ambrose jr and Benjamin. They lived in Division Five of the Township which stretches northwest from Dufferin and Eglinton to the Humber River in the vicinity of Weston.
In Mitchell's Directory, compiled in 1863, Ambrose Kent was a grocer at 288 Yonge Street, i.e., in the middle of today's Eaton Centre. Ambrose jr and Benjamin were with him--Ambrose jr still didn't have an occupation, but Benjamin was now a watchmaker. John Kent (not found in the census) appeared next door at 286-1/2. There is a Joseph Kent on Walton Street (north of Gerrard but easy walking distance from Yonge and Queen). No other people of the surname Kent appear in the Directory.
There were two other Kent households in York Township, once again in Division Five. William, age 28, was another bricklayer, and F G Kent, described as a gentleman, age 47, with a wife nine years older than himself. Was this couple the source of capital for Ambrose's move?
Subscribe to:
Posts (Atom)
Posts
