Question about AI technologies to solve this "learning" challenge

curiouswebster
curiouswebster used Ask the Experts™
on
Question about AI technologies to solve this "learning" challenge

I am new to AI and have had no real experience coding anything related to Natural Language Processing. But, I am eager to jump in!

I need to find a method that I can find the following two options have a high match:

#1 Frank's First Car was a Red Chevy

#2
Frank has never had more than one car
Frank would only buy a red car
Frank comes from a family that only buys Chevy's

#3
Frank has never owned a car

Are there a combination of AI technologies which can:
1) parse these statements
2) preserve them in some fashion
3) Allow them to be organized such that the comparison between #1 vs. #2 returns a very high match (90+%) versus #2 vs. #3

I am interested to see if such tool and languages exist for this.

Thanks
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Data Warehouse / Database Architect
Commented:
I don't know of any publicly available AI software, much less any that will do what you want.

And it's a lot more complicated to program than it might appear at first glance.  Given the rules as a set of equations, the solution is nearly trivial.  But you need the process to convert those rules to equations and that requires that the AI software have an understanding of the English language.

>Frank's First Car was a Red Chevy
>Frank has never had more than one car

The first line requires an knowledge of car manufacturers and their abbreviations and nicknames.  To you and me it's trivial.  But what if the name was Rbicel?  We don't know what that is so we derive from the rest of the statement that it's a car.  We also have to understand the grammar to know that the possessive case of Frank (Frank's).  It must also be able to resolve ambiguities.  What if the subject was Henry Ford instead of Frank?  "Ford's First Car was a Red Chevy".  A simple substitution might result in "Car's First Car was a Red Car".  Oops.  And the misuse of capitalization is another challenge.  "Frank" is the first word of the sentence so it should be capitalized.  It's also being used as a proper noun.  The AI must understand that grammar to know that in the context of that sentence "Frank" is a proper noun, not an adjective.

There's a lot involved in parsing that simple sentence.  It's pretty amazing that we "just know" from hearing the sentence exactly what it means.  It will take a lot of programming for any AI system to fully understand it, too.
curiouswebsterSoftware Engineer

Author

Commented:
Thanks for your insight.  But I hope I am not the first person to ask this question.

Can you think of languages best suited for this kind of challenge?

Python?

And what about neural networks? Any language best suited?

Or will I need to dust off my C++?

Or...

Throw in the towel and outsource it to LUIS...

https://docs.microsoft.com/en-us/azure/cognitive-services/luis/what-is-luis
Kent OlsenData Warehouse / Database Architect
Commented:
Building such an engine on a level to handle anything at a level above trivial is probably beyond the ability of a single programmer.  Remember that parse English language text and understand the context of every word in the sentence.  That's a huge task that many grammar checkers don't even get correct.  (AI might help grammar checkers to be better!)

If you can use something like LUIS where you define your rules and not be concerned with the language issues you're a long way ahead of the game.  (I've never used LUIS and don't know if there are other/better options out there.)
curiouswebsterSoftware Engineer

Author

Commented:
I have access to a team, if I can understand the problem domain well enough to put together a plan.

Thanks
I would look at some of the big Open source library for NLP first - like OpenNLP from Apache:
https://opennlp.apache.org

That should let you do:
>> 1) parse these statements
>> 2) preserve them in some fashion

as it will let you parse the data into both a tree structure (for the structure of the sentence) and give you the underlying dictionary entry for each word (so you can connect that "buy" and "buys" are both from the same root word).

 Now as for
>> 3) Allow them to be organized such that the comparison between #1 vs. #2 returns a very high match (90+%) versus #2 vs. #3

This is where things get a lot harder.  Because here you're looking for a tool that makes inferences based on a lot of knowledge of the world.
That doesn't exist "in general" (because then we'd have solved the General AI problem and Turing test which today remains unsolved).

What you can do however is develop a knowledge base in a single area - in this case car buying and car ownership and then you can make these sorts of inferences.  If you use OpenNLP (which is in Java) then you might take a look at JESS (https://jessrules.com/) which is an expert system toolkit written in Java.

Between those two you may be able to build what you need.  But don't expect this to be an off-the-shelf, just plug it together and be done in a week sort of thing.  This will take work to build, but it can be done.

Doug

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial