Skip to content

Python: Implement module ClickHouseDriver.qll - #5889

Merged
RasmusWL merged 4 commits into
github:mainfrom
japroc:python-clickhouse-driver
May 25, 2021
Merged

Python: Implement module ClickHouseDriver.qll#5889
RasmusWL merged 4 commits into
github:mainfrom
japroc:python-clickhouse-driver

Conversation

@japroc

@japroc japroc commented May 12, 2021

Copy link
Copy Markdown
Contributor

Hello!

The idea of this PR is to improve default SQLInjection query by adding ClickHouseDriver.qll module with describes clickhouse-driver PyPI package. This module hsa self Client with several methods to execute query. Also clickhouse-driver has a PEP249 interface.

When this module will be moved from experimental folder to standart library, it would automatically improve existing Sqlnjection.ql query. But for now, for a demonstration, i created a separate query which imports experimental module and repeats standart query. I also added python snippet for clickhouse-driver usage and qhelp which describes different cases from that example code.

Thanks,

Evgenii.

@japroc
japroc requested a review from a team as a code owner May 12, 2021 19:47
@japroc japroc changed the title [C++] Implement module ClickHouseDriver.qll [Python] Implement module ClickHouseDriver.qll May 12, 2021
@japroc japroc changed the title [Python] Implement module ClickHouseDriver.qll Python: Implement module ClickHouseDriver.qll May 13, 2021

@intrigus-lgtm intrigus-lgtm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some simple comments.
I did not look at the qll file.
(Note that I'm not a member of the github/codeql team)

Comment thread python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.py Outdated
Comment thread python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.py Outdated
Comment thread python/ql/src/experimental/Security/CWE-089/ClickHouseSQLInjection.qhelp Outdated
</p>

<p>
In the first case, the query executed via aioch Client. aioch - is a module

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know query help files should not contain too many examples.
I think one/two "BAD" cases (maybe one aioch case and one `Client class case) and one "GOOD" case should be enough.

@RasmusWL RasmusWL left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @japroc, thanks for this contribution 👍

I'll take a deeper look at reviewing your code tomorrow.

From a very superficial look, I can see that you've added a new query in this PR. While that's not necessarily a bad idea for local development, to be able to test out your changes, I think we'll delete that after promoting your modeling out of experimental. If there are any of the textual comments in qhelp file that are important to keep, I would like to request that you add these to python/ql/test/experimental/semmle/python/frameworks/clickhouse-driver/ClickHouseDriver.py instead 😊

Very nice to see python/ql/test/experimental/semmle/python/frameworks/clickhouse-driver/ClickHouseDriver.ql, we usually use a slightly different approach with ConceptsTest.ql and writing expectations inline with special comment syntax (which you can see in action here), but I think your test does the job pretty nicely as well 👍

@RasmusWL RasmusWL left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this code looks good to me 👍 (good job)

I have one question on client subclasses, but I don't think anything is a major blocker for us merging this PR 👍 (I might do a few minor stylistic cleanups when promoting this out of experimental)

Comment on lines +38 to +42
private DataFlow::Node client_ref() {
result = clickhouse_driver().getMember("Client").getASubclass*().getAUse()
or
result = aioch().getMember("Client").getASubclass*().getAUse()
}

@RasmusWL RasmusWL May 21, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part surprised me. Do you have an example of someone who provides their own Client subclass in a real project? (or is this just something that you thought might happen and you wanted to capture?)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sure i have seen few projects where there are custom subclasses of clickhouse-driver's Client. I did not save links to these projects. And actually dont remember how popular these projects are. I can spend some time on weekends to find them. I will put links here. And we will resolve this moment further.

What do you think? :)

@japroc japroc May 21, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, thanks for providing those examples 👍 having seen those, I think that it makes good sense to have support for that usage pattern 👍

@japroc
japroc requested a review from RasmusWL May 24, 2021 14:33

@RasmusWL RasmusWL left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks really good 💪

This needs a few minor adjustments after being merged, but I'm happy to do that myself 👍

(tests needs to complete, then I can merge)

@RasmusWL
RasmusWL merged commit 35793a1 into github:main May 25, 2021
@japroc

japroc commented May 25, 2021

Copy link
Copy Markdown
Contributor Author

Thanks, @RasmusWL! Gonna celebrate my very first PR to codeql 😄
And i will watch for further adjustments to learn something and make future queries more efficient!

Have a good day!

@RasmusWL

Copy link
Copy Markdown
Member

Nice 🎉 I did some adjustments in #5950, so when that is merged, your contributions will be part of our standard analysis 💪

@japroc

japroc commented May 27, 2021

Copy link
Copy Markdown
Contributor Author

Nice 🎉 I did some adjustments in #5950, so when that is merged, your contributions will be part of our standard analysis 💪

Wow, that was really quick. I am happy 😊 Thank you, @RasmusWL!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants