Comments on: Language Detection

By: James

James — Tue, 23 Mar 2010 09:52:00 +0000

@Tushar,

Thanks for the JSON parser ... which makes life much easier

Cheers

By: Tushar Mehta

Tushar Mehta — Mon, 15 Mar 2010 21:08:00 +0000

Hi Dick:

The Google response is in JSON. Rather than parse the result yourself, get one http://www.ediy.co.nz/vbjson-json-parser-library-in-vb6-xidc55680.html or http://code.google.com/p/vba-json/

I used the latter since it is Excel based VBA.

Now, the function becomes

Public Function DetectLanguage(sInput As String, Optional dConfidence As Double = 0.25) As String

Dim clsLanguage As New cLanguage
With clsLanguage
.InputText = sInput

DetectLanguage = IIf(.Confidence > dConfidence, .LanguageCode, "unknown")
End With
End Function

In the cLanguage class, modify:

Private msInputText As String, ResponseText As String, ResponseObj

If oHttp.Status = 200 Then
ResponseText = oHttp.ResponseText
Dim X As JSONLib: Set X = New JSONLib
Set ResponseObj = X.parse("[" & ResponseText & "]")
End If

Public Property Get Confidence() As Double
Confidence = ResponseObj(1)("responseData")("confidence")
End Property
Public Property Get LanguageCode() As String
LanguageCode = ResponseObj(1)("responseData")("language")
End Property

and the code to use DetectLanguage

Sub testDetectLang()
MsgBox DetectLanguage("this is one language") _
& "," & DetectLanguage("ich bien ein berliner") _
& "," & DetectLanguage("esta es una segunda lengua") _
& "," & DetectLanguage("eto mae hyn yn iaith arall") _
& "," & DetectLanguage("Bu baska bir dildir", 0) _
& "," & DetectLanguage("Dit is nog 'n ander taal")
'Welsh is reported as en, Afrikaans is reported as nl, _
tr confidence is lower than the default, don't know what it is but I used 0
End Sub

By: eyeIT

eyeIT — Mon, 15 Mar 2010 20:54:00 +0000

Indeed, we Swiss people do have 4 official national languages and it includes Rumantsch.

Another way crossed my mind, but it perhaps fulfills not quite the task that was asked for. I would simply retrieve this information by

Application.LanguageSettings.LanguageID(msoLanguageIDUI)

This way, you would get the language used. Of course, it is not a detection which you could guess a language independently from the used system (that's maybe what was asked for).

Anyway, it's a very interesting solution you suggested.

By the way, can anyone explain me what reason you could have to use Python for Excel?

By: Dick Kusleika

Dick Kusleika — Mon, 15 Mar 2010 19:51:00 +0000

I read an rss feed from WordPress.com for any post that contains the word ‘Excel’. It gets tons of foreign language blogs since Excel is the same in every language. So I run them through an rss filter that filters out based on one word. I just find a word in the post that couldn’t possibly appear in English and use that in my filter. It’s the complement to what we’re doing here; excluding rather than including languages. But I think it supports what fzz says; it won’t take many carefully chosen words to identify a language provided the input is long enough.

By: fzz

fzz — Mon, 15 Mar 2010 19:01:00 +0000

I'd figure the present tense forms of the verbs to be and to have along with the pronouns would be sufficient to identify any German or Latin root language. For that matter, I'd figure the frequency of the German articles der and das alone would be sufficient to identify German from either French or Italian. Then the frequency of French plural definite article les and the cojunction et should be sufficient to identify French from Italian.

IOW, I'd be willing to bet the following would be sufficient in any noncontrived text of 50 words or more.

German der, das, ein, und, oder
French les, un, et, ou

Tangent: I thought Rumantsch was still also a national language.

By: mt

mt — Mon, 15 Mar 2010 15:17:00 +0000

Related to this, here's a function for translating text via the Google Translate API: http://mikaelspage.blogspot.com/2009/09/excel-function-for-translating-text.html