【Discuss】Run examples automatically when raise a PR

[Erlu Chen]

Background:

Now when we raise a PR, CI build will not run examples, such as CarbonSessionExample.

But in many cases, our modification may influence examples, so I raise a discuss about whether CI build should run examples when raise a PR. If yes, how we can achieve it in a reasonable way.

My idea:

I think CI build should run examples when raise a PR.
One way to achieve this is : We can make a running examples list in a test case. when CI start to build,
it will run test case, then it will trigger our examples.

This is my rough idea，we can discuss here, any suggestions welcome.

Regards.
Chenerlu

[David CaiQiang]

How to known whether example run successfully or not?

[Erlu Chen]

Hi, David

I come up with two ideas, one is simple check, one is complex check.

Simple check just sets a flag to false before running the example, and reset it to true in the end of the example. If all test function is OK, the flag is true, so just check the flag. Besides, we can check the example log, if the flag is true and no "error" in its log file, then we can assert that this example is successful.

Complex check means we need match file for each test point, this will be more effective to know whether the example is successful or not. But the disadvantage is many match file in the project which is not a good thing for carbondata I think.

So I suggest simple check, any idea ?