ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Needham <needb...@gmail.com>
Subject Re: Access a cache loaded by DataStreamer with SQL
Date Thu, 14 Mar 2019 19:59:41 GMT
Hi,

Here is the code I am using

package org.apache.ignite.examples;
import java.util.Collections;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.IgniteDataStreamer;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.QueryEntity;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;

public class LoadTableWithDataStreamer {
    public static void main(String[] args) {
        try (Ignite ignite =
Ignition.start("E:\\ignite\\apache-ignite-2.7.0-src\\examples\\config\\example-ignite.xml"))
{
            IgniteCache<Object, Object> personCache =
ignite.getOrCreateCache(new CacheConfiguration<>("PUBLIC")
                .setQueryEntities(Collections.singleton(
                    new QueryEntity(Integer.class,
String.class).setTableName("person_table"))));

            IgniteCache<Object, Object> placeCache =
ignite.getOrCreateCache(new CacheConfiguration<>("PUBLIC")
                    .setQueryEntities(Collections.singleton(
                        new QueryEntity(Integer.class,
String.class).setTableName("place_table"))));
            try (IgniteDataStreamer<Object, Object> ds =
ignite.dataStreamer("person")) {
                ds.addData(1, "John");
            }

            System.err.println("Query result");
            personCache.query(new SqlFieldsQuery("select * from
person_table")).getAll().forEach(System.err::println);
        }
    }
}

I want to create to queryable tables that I can load using the data
streamer.  The config is the default config in the java examples


On Mon, Mar 11, 2019 at 3:06 AM Ilya Kasnacheev <ilya.kasnacheev@gmail.com>
wrote:

> Hello!
>
> Can you perhaps post your config and code? Much easier than writing my own
> boilerplate.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 8 мар. 2019 г. в 00:36, Mike Needham <needbrew@gmail.com>:
>
>> Would it be possible for someone to provide a sample that uses the
>> DataStreamer for multiple large caches and the resulting caches are
>> queryable from ODBC tools like Tableau and DBeaver?  I can get one to work,
>> but cannot get the cache naming to work for a second one.
>>
>> On Thu, Mar 7, 2019 at 7:53 AM Ilya Kasnacheev <ilya.kasnacheev@gmail.com>
>> wrote:
>>
>>> Hello!
>>>
>>> JDBC with SET STREAMING ON should work reasonably well. You could also
>>> use CacheStore.
>>>
>>> With .Net I guess you will have to stick to DataStreamer & learn how to
>>> integrate it with Indexing properly. Or use JDBC from .Net.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> чт, 7 мар. 2019 г. в 16:31, Mike Needham <needbrew@gmail.com>:
>>>
>>>> Is that the most efficient way to load millions of rows from a database
>>>> table into a cache? I guess I am not seeing how this would work for
>>>> millions of rows in the source database.  That also appears to be a JAVA
>>>> only solution and it would be preferable to have it be usable for .NET as
>>>> well as java and hopefully python.
>>>>
>>>> On Thu, Mar 7, 2019 at 6:55 AM Ilya Kasnacheev <
>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>> What prevents you from reading rows from DB and feeding them to JDBC
>>>>> prepared statement?
>>>>>
>>>>> Regards,
>>>>> --
>>>>> Ilya Kasnacheev
>>>>>
>>>>>
>>>>> чт, 7 мар. 2019 г. в 15:51, Mike Needham <needbrew@gmail.com>:
>>>>>
>>>>>> And what if the data is already in a database table?  I do not want
>>>>>> to read from the table to write to a file to load a cache.
>>>>>>
>>>>>> On Tue, Mar 5, 2019 at 4:06 AM Ilya Kasnacheev <
>>>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>>>
>>>>>>> Hello!
>>>>>>>
>>>>>>> The preferred approach is to use Thin JDBC client:
>>>>>>> https://apacheignite-sql.readme.io/docs/jdbc-driver
>>>>>>>
>>>>>>> Regards,
>>>>>>> --
>>>>>>> Ilya Kasnacheev
>>>>>>>
>>>>>>>
>>>>>>> пн, 4 мар. 2019 г. в 19:39, Mike Needham <needbrew@gmail.com>:
>>>>>>>
>>>>>>>> Thanks for the links,  If SET STREAMING ON is the preferred
method,
>>>>>>>> how would you do this in code rather than from a file with
all the insert
>>>>>>>> statements.
>>>>>>>>
>>>>>>>> On Mon, Mar 4, 2019 at 1:44 AM Ilya Kasnacheev <
>>>>>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> You can see at this page:
>>>>>>>>> https://apacheignite-sql.readme.io/docs/sql-and-key-value-usage
>>>>>>>>>
>>>>>>>>> And then substitite cache.put() with addData() and cache
name will
>>>>>>>>> be SQL_PUBLIC_{table name in caps}
>>>>>>>>>
>>>>>>>>> There are not many examples since this is being discouraged
in
>>>>>>>>> favor of SET STREAMING ON, actually:
>>>>>>>>> https://apacheignite-sql.readme.io/docs/set
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> --
>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> пт, 1 мар. 2019 г. в 22:48, Mike Needham <needbrew@gmail.com>:
>>>>>>>>>
>>>>>>>>>> I have looked at the documentation and the code samples
and
>>>>>>>>>> nothing is doing what I am trying to do.  I want
to be able to use the
>>>>>>>>>> datastreamer to load 3 or 4 TABLES in a cache for
an application that we
>>>>>>>>>> use.  If I create the tables using a create table
syntax how do attach a
>>>>>>>>>> datastreamer to the different caches if the cache
name is PUBLIC for all of
>>>>>>>>>> them?
>>>>>>>>>>
>>>>>>>>>> On Thu, Feb 28, 2019 at 8:13 AM Ilya Kasnacheev <
>>>>>>>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello!
>>>>>>>>>>>
>>>>>>>>>>> I have linked the documentation page, there are
also some code
>>>>>>>>>>> examples in distribution.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> --
>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> чт, 28 февр. 2019 г. в 17:10, Mike Needham
<needbrew@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Is there any examples that show the steps
to do this
>>>>>>>>>>>> correctly?  I stumbled upon this but have
no idea if it is the best way to
>>>>>>>>>>>> do this
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Feb 28, 2019 at 6:27 AM Ilya Kasnacheev
<
>>>>>>>>>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello!
>>>>>>>>>>>>>
>>>>>>>>>>>>> There's no restriction on cache name
but setting it up for the
>>>>>>>>>>>>> first time may be tricky indeed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ср, 27 февр. 2019 г. в 19:48,
needbrew99 <needbrew@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> OK, was able to get it working. 
Apparently the cache name
>>>>>>>>>>>>>> has to be PUBLIC
>>>>>>>>>>>>>> and it will create a table based
on the object definition
>>>>>>>>>>>>>> that I have.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Some days it just not worth chewing through
the restraints*
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Some days it just not worth chewing through the
restraints*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Some days it just not worth chewing through the restraints*
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Some days it just not worth chewing through the restraints*
>>>>>>
>>>>>
>>>>
>>>> --
>>>> *Some days it just not worth chewing through the restraints*
>>>>
>>>
>>
>> --
>> *Some days it just not worth chewing through the restraints*
>>
>

-- 
*Some days it just not worth chewing through the restraints*

Mime
View raw message