RDFGS 2.0: Data Aggregation & Sources... Need Your Help! (1 Viewer)

xydrine

Vengeance. Justice. Fire and Blood.
Advanced User
Lifetime Premium Member
Joined
Oct 28, 2010
Messages
26,932
Reaction score
23,965
Location
/dev/null
Hey everyone,

So as you know, shortly after the RDFVPN stuff is finished I'll be switching into high gear when it comes to the RDFGS 2.0.

The RDFGS 2.0 is going to be a massive project and is going to be closed alpha/beta-tested by those of you who are interested in doing so. It's more or less going to be a completely on-going project.

While I am still drafting up a huge proposal for the project to present to you guys to kick things off, I am doing a massive amount of planning as well. Some parts go into more detail than others, and this project is going to be done right - from beginning to end, from planning to development to life-cycle to support, like a normal large-scale professional coding project would.

Anyway, I wanted to start having you guys submit here in this thread details on data sources that should be integrated into the RDFGS 2.0. Examples may be something like 'the red light camera database that came with the Whistler XX2233' or something like 'the red light camera database at www . blah . com' so forth and so on.

Paid sources are fine, too! We will integrating a ton of different sources and I expect some of them will be premium/cost money to get a hold of. That's fine, the more sources the better.

So anyway, if you guys can start submitting sources that you'd like to see integrated, or even ones that you know about that someone might find useful, or would add more data points to the project, that'd be great!

Thanks!
 

drtoddw

Jammer bieten keinen 100 %-Schutz
Advanced User
Premium Member
Joined
Nov 3, 2016
Messages
3,815
Reaction score
5,056
Location
Behind the wheel
Last edited:

xydrine

Vengeance. Justice. Fire and Blood.
Advanced User
Lifetime Premium Member
Joined
Oct 28, 2010
Messages
26,932
Reaction score
23,965
Location
/dev/null

dchemist

Lifetime RDF Contributor
Advanced User
Lifetime Premium Member
Joined
Jul 26, 2017
Messages
2,985
Reaction score
5,421
Location
Benton, AR
Some of use are working on broadcast (transmit) frequencies of LEO radio. Further done the road, there is quite a bit of information regarding signal triangulation so that we can get an idea of where the transmission is coming from. I feel like SDRs/Raspberry Pi incorporation into a mobile unit is completely doable. I think this is worth watching for further development and possible incorporation.

www.rdforum.org: Location via Repeater Input
 

thebravo

Security Detachment
RDF Intelligence Agency
Advanced User
Lifetime Premium Member
Joined
Dec 31, 2016
Messages
5,294
Reaction score
9,970
Location
FL (formerly CT)
Integrate the AVR that we have been working on here on RDF (aircraft vascar), perhaps even a way to incorporate ADSB flight data so you can see when an aircraft is up flying? some of that you might be able to pull off ADSBexchange, flightaware,FR24 etc and filter down to aircraft of interest by state, but might also be good to have a way to stream in ADSB data from members that run their own Piaware or similar receivers as some area's have blocked tail numbers from tracking sites so the data might be already filtered to not include flights we want to know about. I'm sure there will a way for folks to submit local intel, common trap areas, lidar gun types encountered, unmarked Vehicles spotted, radar bands encountered etc. I have a log I have been keeping for my area with threats I have found.
 

dchemist

Lifetime RDF Contributor
Advanced User
Lifetime Premium Member
Joined
Jul 26, 2017
Messages
2,985
Reaction score
5,421
Location
Benton, AR
Integrate the AVR that we have been working on here on RDF (aircraft vascar), perhaps even a way to incorporate ADSB flight data so you can see when an aircraft is up flying? some of that you might be able to pull off ADSBexchange, flightaware,FR24 etc and filter down to aircraft of interest by state, but might also be good to have a way to stream in ADSB data from members that run their own Piaware or similar receivers as some area's have blocked tail numbers from tracking sites so the data might be already filtered to not include flights we want to know about. I'm sure there will a way for folks to submit local intel, common trap areas, lidar gun types encountered, unmarked Vehicles spotted, radar bands encountered etc. I have a log I have been keeping for my area with threats I have found.
x2

Posted from my Pixel 2 using the RDF Mobile App!
 

johnboy00

Geaux Tigers!
Advanced User
Lifetime Premium Member
Joined
Sep 6, 2016
Messages
3,733
Reaction score
6,872
Location
Raleigh, NC
Thanks, this information is definitely helpful. I know other people know websites or products that access databases, please share the names of them or link them to me! Data aggregation is important to this project!

Posted from my SM-N960U using the RDF Mobile App!
JBV1 can import RLC and speed cam data from poi-factory.com, but their terms of service would likely be a problem for RDFGS 2.0.
 

Transporter

ModWight Transporter
Advanced User
Lifetime Premium Member
Joined
Jul 3, 2018
Messages
2,578
Reaction score
3,723
Location
In front of you but behind a Rabbit
What type of Beta Testers are you looking for? What can Forum Members help with/work on/build/gather to help with the project?
 

Vancity23

(Prev. LexusISF)
Advanced User
Premium Member
Joined
Jun 11, 2017
Messages
3,465
Reaction score
3,948
Location
Vancouver, B.C.
Dont know if there is anything I can do to help, but just want to offer...
 

fitz4321

Running With Scissors
Advanced User
Premium Member
Joined
Dec 12, 2015
Messages
2,155
Reaction score
3,886
Location
Bay Area, CA
I’ll state the obvious, Waze info.

There is an app called Radarbot. I haven’t tested it yet, but it looks to have a decent amount of users. Perhaps the information there could be accessible. Maybe I’ll run it in the background for a while to see if there is anything useful there.

Also, count me in for testing.
 
Last edited:

xydrine

Vengeance. Justice. Fire and Blood.
Advanced User
Lifetime Premium Member
Joined
Oct 28, 2010
Messages
26,932
Reaction score
23,965
Location
/dev/null
Awesome input, gentlemen. Thank you!

So let me get caught up on reading/responding to everything...

Some of use are working on broadcast (transmit) frequencies of LEO radio. Further done the road, there is quite a bit of information regarding signal triangulation so that we can get an idea of where the transmission is coming from. I feel like SDRs/Raspberry Pi incorporation into a mobile unit is completely doable. I think this is worth watching for further development and possible incorporation.

www.rdforum.org: Location via Repeater Input
Cool cool, so I don't know how far you guys have dug into what I've brainstormed for the RDFGS 2.0 project, and honestly it's going to really be a project completely on its own from RDFGS 1.0 - a brand new project which will happen to incorporate RDFGS 1.0 data into it - but for the moment we'll just keep referring to it as RDFGS 2.0... but I am 100% serious about wanting to have a device which interacts with/assists/etc RDFGS 2.0 functionality released along with the software side of things, and very very early brainstorming sessions actually had me thinking about using a raspberry pi as the foundation for it. Going with a more targeted/even a custom devboard will probably be the best way to go about it, but just figured I'd mention this because you guys did...

...ps, pardon my somewhat inability to form proper sentences at the moment, its 3am and ive been awake for about 30 hours, lol...

...but anyway. My idea for the RDFGS 2.0 device has evolved significantly since I started out thinking about it. There are so many unique devboards out there. I've actually been thinking about using a white-label MVNO service to give the device a SIM card which would give itself its own internet connection, and limit the device to connect to only RDFGS 2.0 servers for data transmission, etc. But these ideas are very very far ahead into the future. The software side of the RDFGS 2.0 will be pretty much done being coded before we start looking at the hardware side of it.

But yes, I do like the idea of using LEO radio frequencies against them. It's just another layer of data that could be rather helpful on many levels.

Integrate the AVR that we have been working on here on RDF (aircraft vascar), perhaps even a way to incorporate ADSB flight data so you can see when an aircraft is up flying? some of that you might be able to pull off ADSBexchange, flightaware,FR24 etc and filter down to aircraft of interest by state, but might also be good to have a way to stream in ADSB data from members that run their own Piaware or similar receivers as some area's have blocked tail numbers from tracking sites so the data might be already filtered to not include flights we want to know about. I'm sure there will a way for folks to submit local intel, common trap areas, lidar gun types encountered, unmarked Vehicles spotted, radar bands encountered etc. I have a log I have been keeping for my area with threats I have found.
Oh absolutely. What I'm thinking is that for the RDFGS 2.0 application, similar to Google Maps or whatever, you will be able to add/remove different map layers to the live map screen dashboard, which would have different selectors (i.e. in Google Maps or whatever you can add weather, terrain, satellite image, etc) for different data elements like red light cams, vascar lines, common radar trap locations, etc.

As far as live data goes, that's awesome and 100% something that we should look into adding if the data exists/or if we have the ability to make it exist (I don't know sh!t all about aircraft speed enforcement at the moment) because, AND I CAN'T STRESS THIS ENOUGH, live data/the live map/live map dashboard is going to be the main attraction for the RDFGS 2.0. Not only will the RDFGS 2.0 have historical data (we are going to save EVERYTHING we possibly can, as much data as possible, no MB/GB/TB storage limits, we want it all, forever) but it will work similarly to how Waze works where the live map is what, like I said 'main attraction', everyone comes to see! The RDFGS 2.0 is literally going to be everything we have dreamed of in a Waze-like app - BECAUSE WE ARE THE ONES BUILDING IT!

JBV1 can import RLC and speed cam data from poi-factory.com, but their terms of service would likely be a problem for RDFGS 2.0.
Honestly I am not really concerned about TOSes, one way or another if there's a data source out there, I want it in RDFGS 2.0 ;).

For the sake of development and brainstorming/planning, let's proceed under the assumption that other TOSes and etc out there are irrelevant.

What type of Beta Testers are you looking for? What can Forum Members help with/work on/build/gather to help with the project?
Dont know if there is anything I can do to help, but just want to offer...
Don't worry, you guys will definitely be read-in to everything RDFGS 2.0 related. There will be substantial announcements created when the time comes and I'm not going to be turning away our existing members for alpha/beta testing etc. RDF is an enthusiast community and the RDFGS 2.0 is going to be created for the enthusiast (of course it will still be usable by all people of all skill levels but the enthusiast is at the heart of this application), by the enthusiasts.

As far as bringing the community in to start helping/building/gathering/etc, well... some of that has started in its very very most preliminary phases (i.e. this thread). I'm probably going to create an RDFGS 2.0 section here on RDF purposely for the project, to keep things separate from RDFGS 1.0 (which I should have back up by the end of this weekend - finishing up the XenForo integration part of it so you guys will at least be able to access that again soon). Actually I'll go ahead and do that in a few minutes.

I’ll state the obvious, Waze info.

There is an app called Radarbot. I haven’t tested it yet, but it looks to have a decent amount of users. Perhaps the information there could be accessible. Maybe I’ll run it in the background for a while to see if there is anything useful there.

Also, count me in for testing.
Thanks for the heads up about radarbot, interesting app - definitely has a ton of users. That kind of gives me hope and frustration at the same time, lol. On one hand it's clear there's a general demand for something like this, on the other hand its clear there's some "competition" - but of course, when we go live, NOTHING will compare or compete with what we are creating.

This project - RDFGS 2.0 (again, we've gotta get a better name for this damn thing, lol) - and I've talked a little bit about this here and there, but the size of the project is going to be massive. It's literally going to be my "career" (and my one and only "job") if you will, once launched. I'm stating this to show you guys how important/big/life changing (for me) this project is going to be - it's not going to be a "side project" or something I'll be coding in my spare time, etc.

Anywho, I'm going to go create that RDFGS 2.0 section now. I'll do some thread cleanup and this thread will end up in that new section soon as well.
 

dchemist

Lifetime RDF Contributor
Advanced User
Lifetime Premium Member
Joined
Jul 26, 2017
Messages
2,985
Reaction score
5,421
Location
Benton, AR
Awesome input, gentlemen. Thank you!

So let me get caught up on reading/responding to everything...



Cool cool, so I don't know how far you guys have dug into what I've brainstormed for the RDFGS 2.0 project, and honestly it's going to really be a project completely on its own from RDFGS 1.0 - a brand new project which will happen to incorporate RDFGS 1.0 data into it - but for the moment we'll just keep referring to it as RDFGS 2.0... but I am 100% serious about wanting to have a device which interacts with/assists/etc RDFGS 2.0 functionality released along with the software side of things, and very very early brainstorming sessions actually had me thinking about using a raspberry pi as the foundation for it. Going with a more targeted/even a custom devboard will probably be the best way to go about it, but just figured I'd mention this because you guys did...

...ps, pardon my somewhat inability to form proper sentences at the moment, its 3am and ive been awake for about 30 hours, lol...

...but anyway. My idea for the RDFGS 2.0 device has evolved significantly since I started out thinking about it. There are so many unique devboards out there. I've actually been thinking about using a white-label MVNO service to give the device a SIM card which would give itself its own internet connection, and limit the device to connect to only RDFGS 2.0 servers for data transmission, etc. But these ideas are very very far ahead into the future. The software side of the RDFGS 2.0 will be pretty much done being coded before we start looking at the hardware side of it.

But yes, I do like the idea of using LEO radio frequencies against them. It's just another layer of data that could be rather helpful on many levels.



Oh absolutely. What I'm thinking is that for the RDFGS 2.0 application, similar to Google Maps or whatever, you will be able to add/remove different map layers to the live map screen dashboard, which would have different selectors (i.e. in Google Maps or whatever you can add weather, terrain, satellite image, etc) for different data elements like red light cams, vascar lines, common radar trap locations, etc.

As far as live data goes, that's awesome and 100% something that we should look into adding if the data exists/or if we have the ability to make it exist (I don't know sh!t all about aircraft speed enforcement at the moment) because, AND I CAN'T STRESS THIS ENOUGH, live data/the live map/live map dashboard is going to be the main attraction for the RDFGS 2.0. Not only will the RDFGS 2.0 have historical data (we are going to save EVERYTHING we possibly can, as much data as possible, no MB/GB/TB storage limits, we want it all, forever) but it will work similarly to how Waze works where the live map is what, like I said 'main attraction', everyone comes to see! The RDFGS 2.0 is literally going to be everything we have dreamed of in a Waze-like app - BECAUSE WE ARE THE ONES BUILDING IT!



Honestly I am not really concerned about TOSes, one way or another if there's a data source out there, I want it in RDFGS 2.0 ;).

For the sake of development and brainstorming/planning, let's proceed under the assumption that other TOSes and etc out there are irrelevant.





Don't worry, you guys will definitely be read-in to everything RDFGS 2.0 related. There will be substantial announcements created when the time comes and I'm not going to be turning away our existing members for alpha/beta testing etc. RDF is an enthusiast community and the RDFGS 2.0 is going to be created for the enthusiast (of course it will still be usable by all people of all skill levels but the enthusiast is at the heart of this application), by the enthusiasts.

As far as bringing the community in to start helping/building/gathering/etc, well... some of that has started in its very very most preliminary phases (i.e. this thread). I'm probably going to create an RDFGS 2.0 section here on RDF purposely for the project, to keep things separate from RDFGS 1.0 (which I should have back up by the end of this weekend - finishing up the XenForo integration part of it so you guys will at least be able to access that again soon). Actually I'll go ahead and do that in a few minutes.



Thanks for the heads up about radarbot, interesting app - definitely has a ton of users. That kind of gives me hope and frustration at the same time, lol. On one hand it's clear there's a general demand for something like this, on the other hand its clear there's some "competition" - but of course, when we go live, NOTHING will compare or compete with what we are creating.

This project - RDFGS 2.0 (again, we've gotta get a better name for this damn thing, lol) - and I've talked a little bit about this here and there, but the size of the project is going to be massive. It's literally going to be my "career" (and my one and only "job") if you will, once launched. I'm stating this to show you guys how important/big/life changing (for me) this project is going to be - it's not going to be a "side project" or something I'll be coding in my spare time, etc.

Anywho, I'm going to go create that RDFGS 2.0 section now. I'll do some thread cleanup and this thread will end up in that new section soon as well.
I've dug into it quite a bit and this post sums everything up nicely! Thanks for the explanation, especially the white label MVNO.

Posted from my Pixel 2 using the RDF Mobile App!
 

xydrine

Vengeance. Justice. Fire and Blood.
Advanced User
Lifetime Premium Member
Joined
Oct 28, 2010
Messages
26,932
Reaction score
23,965
Location
/dev/null
Speaking of sources, not exactly the same kind I was talking about in this thread, but sources none the less...

One thing I was thinking about is, if/when we do the physical RDFGS 2.0 device, perhaps choosing a devboard that allows for multiple external inputs would be ideal - that way for detectors and equipment that doesn't have bluetooth or other means of getting data directly into a smartphone/computer, it would now have a way. This would of course increase the overall usefulness of the device itself and allow for automated reporting of that data to the RDFGS 2.0 database server, as well as create a brand new market for development (of apps, etc) and the like.

Just some thoughts/ideas, mainly notes for myself for later on.
 

FoxStang

Sly fox, dumb bunny.
Advanced User
Premium Member
Joined
Oct 12, 2015
Messages
1,003
Reaction score
2,574
Location
PNW
Does Waze have any sort of API that allows third-party apps to submit data in addition to receiving it? I am all for consolidating its function into RDFGS 2.0, but if we aren't able to mark threats then I'd still end up running Waze on a separate device.
 

thesilverbullet

Clemson Tigers
Advanced User
Premium Member
Joined
Jul 11, 2013
Messages
1,853
Reaction score
1,250
Location
Clemson
can you explain the advantage for custom hardware (devboard w/MVNO) vs using using smart phones for the hardware & software? granted you would have to maintain at lease two operating systems - ios and android
 

dchemist

Lifetime RDF Contributor
Advanced User
Lifetime Premium Member
Joined
Jul 26, 2017
Messages
2,985
Reaction score
5,421
Location
Benton, AR
Speaking of sources, not exactly the same kind I was talking about in this thread, but sources none the less...

One thing I was thinking about is, if/when we do the physical RDFGS 2.0 device, perhaps choosing a devboard that allows for multiple external inputs would be ideal - that way for detectors and equipment that doesn't have bluetooth or other means of getting data directly into a smartphone/computer, it would now have a way. This would of course increase the overall usefulness of the device itself and allow for automated reporting of that data to the RDFGS 2.0 database server, as well as create a brand new market for development (of apps, etc) and the like.

Just some thoughts/ideas, mainly notes for myself for later on.
If you could do something along the lines of Raspberry Pi 3+ (multiple USBs, and WiFi) that would be awesome!

Posted from my Pixel 2 using the RDF Mobile App!
 

xydrine

Vengeance. Justice. Fire and Blood.
Advanced User
Lifetime Premium Member
Joined
Oct 28, 2010
Messages
26,932
Reaction score
23,965
Location
/dev/null
can you explain the advantage for custom hardware (devboard w/MVNO) vs using using smart phones for the hardware & software? granted you would have to maintain at lease two operating systems - ios and android
It would only be as a secondary option if people didn't want to use a smartphone as their primary RDFGS 2.0-interfacing device. Completely out of choice, not required.
 

thebravo

Security Detachment
RDF Intelligence Agency
Advanced User
Lifetime Premium Member
Joined
Dec 31, 2016
Messages
5,294
Reaction score
9,970
Location
FL (formerly CT)
@xydrine something for aircraft tracking: opensky-network.org: The OpenSky Network API documentation — The OpenSky Network API 1.4.0 documentation Opensky has a few different API's of unfiltered data perhaps there is a way we can use that and filter it to some aircraft of interest by state to show live if there are aircraft of interest in the air.

As a Side note, how hard would it be to write a simple app in the near term that can parse that data stream for certain aircraft and shoot say an email if a certain aircraft goes up? recently (last couple weeks) FL-HWY-PTRL has started blocking their aircraft from Flightaware and FR24, so I no longer have a good way to know they are up running vascar traps (other than to keep refreshing the web pages on opensky to see if one takes off, open sky has no feature for alerting you when something goes up). If it's not super hard (not a programmer, know nothing about API's) could you create a rough simple app that can parse that stream for a few tail numbers and alert when one is found?
 

xydrine

Vengeance. Justice. Fire and Blood.
Advanced User
Lifetime Premium Member
Joined
Oct 28, 2010
Messages
26,932
Reaction score
23,965
Location
/dev/null
@xydrine something for aircraft tracking: opensky-network.org: The OpenSky Network API documentation — The OpenSky Network API 1.4.0 documentation Opensky has a few different API's of unfiltered data perhaps there is a way we can use that and filter it to some aircraft of interest by state to show live if there are aircraft of interest in the air.
Cool, I made a note of it (in my upcoming RDFGS 2.0 proposal outline/document I've been developing over the past half year or so), thanks!

Speaking of "sky," I also have listed for potential integrations of SkyWarn, as well as RadioReference.

As a Side note, how hard would it be to write a simple app in the near term that can parse that data stream for certain aircraft and shoot say an email if a certain aircraft goes up? recently (last couple weeks) FL-HWY-PTRL has started blocking their aircraft from Flightaware and FR24, so I no longer have a good way to know they are up running vascar traps (other than to keep refreshing the web pages on opensky to see if one takes off, open sky has no feature for alerting you when something goes up). If it's not super hard (not a programmer, know nothing about API's) could you create a rough simple app that can parse that stream for a few tail numbers and alert when one is found?
Can you show me a link to a page for an aircraft that currently does not exist, and then for an aircraft does currently exist?

I haven't at all taken a look at the api/really any of the pages there so you telling me basically how you distinguish for when an aircraft is currently up/when it's not.

Obviously, in the future, integrating this into the RDFGS 2.0 will be a rather easy thing to do, but for the moment I can help you out with this via a script - it will be rather easy to do if you can provide me with the aforementioned information.
 

Users Who Are Viewing This Thread (Users: 0, Guests: 1)

Latest threads

Latest posts

Social Group Activity

Forum statistics

Threads
76,626
Messages
1,167,320
Members
19,689
Latest member
julies311
Top