Common algorithm tokenization classier for whether a period token is a part of a word or the sentence boundary